Continuous AIF assist previous vs new controller

AIF shared steering · lateral control

From lane hypotheses to a continuous belief about the driver's goal

The MHC tracking condition asks that the system respond to the driver's intentions (Santoni de Sio & van den Hoven, 2018), and the outline proposes the Sense of Agency, measured by intentional binding (Haggard et al., 2002) and the SoA scale (Tapal et al., 2017), as its empirical proxy. Active Inference gives the mechanism: a User Schema that infers the driver's latent intention by minimising free energy, and acts to bring about what that intention predicts (Friston, 2010; Murray-Smith et al., 2025).

Previous controller

  • Belief: lane log-odds, a leaky integrator of the steering column
  • Force: a proportional pull towards a blended target
  • Lanes hard-coded; the belief decays back to the starting lane

New controller

  • Hidden state: the driver's intended lateral position g, continuous, with no lane labels or hypotheses
  • Lanes enter as an environment-informed preference prior, which can be relearned
  • An exact Bayesian filter over g, a user model of how drivers steer, and precision-weighted action

Test suite (live)

running…

Run in your browser on the experiment's own physics and controllers (Test suite slide).

Navigate with ← / → or the menu. The controller is src/simulation/aif_controller.js, the code the experiment runs; every equation can be expanded to show that code.

Design goals

Each goal comes from the theory or from the experimental design, and each is tested by the scenarios on the Test suite slide (tags G1–G7).

G1 Track the driver's intention

MHC tracking condition (Santoni de Sio & van den Hoven, 2018); the outline's User Schema, minimising free energy over q(Ih)

New: a posterior q(g) over the intended position, updated with a generative model of how a driver with goal g steers (Eq. 4–8).

Previous: a leaky integrator of the column input on lane log-odds, not a likelihood.

G2 Tell corrections and noise from intention changes

Outline, Phase 2 (dot-control paradigm after Wen et al., 2015): distinguish motor noise from a genuine change of target

New: precision-weighted evidence (Eq. 5) against a prior with a barrier between the lanes. Brief or scattered input is absorbed; sustained input carries the belief over the barrier.

Previous: integrated the column input, drift noise included.

G3 Distal reasons override proximal ones, only in conflict

Proximity scale of reasons (Mecacci & Santoni de Sio, 2019); preregistration H3b–H3c

New: the obstacle is a collision-footprint hazard on g, acting at a fast rate (Eq. 3, 6). It removes belief only where a collision would occur.

Previous: the sensor pushed the lane log-odds; with a passive driver it collided.

G4 Firm when confident, yielding when uncertain

H-metaphor (Flemisch et al., 2003); Murray-Smith et al. (2025), App. D.1

New: the driver's input passes through directly in proportion to the belief's uncertainty (Eq. 10), optionally also its surprise.

Previous: a fixed heuristic that blended the steering into the target.

G5 No hard-coded intentions

Outline: intentions as preference priors, "probabilistic constraints over expected outcomes"; Murray-Smith et al. (2025) §3.5

New: lanes enter only through the sensed geometry of the prior (Eq. 1), which can be relearned from experience (Eq. 12).

Previous: two lanes and a fixed starting lane built into the belief.

G6 A fair comparison between conditions

Experimental design: conditions should differ in inference, not in plant or raw authority

New: same plant, noise, sensor range and actuator limit as naive; a held key changes lane at least as fast as in the naive condition.

Previous: same plant; authority not checked.

G7 Auditable and replayable

Data quality: trajectories are reconstructed from logged inputs

New: deterministic, with every method tagged by its equation number; the suite and tests/replay_determinism.mjs check exact replay.

Previous: deterministic, but the maths in the plan and the code had diverged.

What the previous AIF controller did

  • Force: a single quadratic well around a blended target. Without steering or an obstacle it behaved like a proportional controller (Πx = 3.6). The double-well prior was drawn in the debug overlay but did not drive the force.
  • Belief: a leaky integrator of the column input (driver plus noise) on lane log-odds, not a likelihood (G1, G2).
  • Decay: the log-odds relaxed towards a baseline lane that was never updated, so after a completed change the belief drifted back to the start lane (G1).
  • Hypotheses: the plan described four discrete hypotheses (StayLeft, StayRight, ChangeLeft, ChangeRight); the code had two lanes hard-coded throughout (G5).

Live runs on the same plant: the previous controller (from a frozen copy, demos/aif_proposal/legacy_controllers.js), the naive PD and the new controller.

Change lane (driver steers for 3 s), then no input for 12 s
0.8 s press, then release

The driver's input is an observation

In Murray-Smith et al.'s semi-autonomous driving vignette (2025, App. D.1), the system sets the steering and observes the driver through the wheel and pedals. The outline's User Schema has the same structure: user inputs are observations oS crossing the system's Markov blanket, and the system's steering is its action uS. This extends Active Inference body-schema work in robotics (Lanillos & Cheng, 2018) from self-perception to perception of the other agent.

Driver goal g (hidden) assist (Markov blanket) sense: u, x, v, obstacle belief q(g) preference prior act: steering force Car & road x, v, obstacle keys driver sees the road

Why mediated steering. The thesis claim is that SoA can be preserved even when proximal (motor) control is automated, provided the system tracks the distal intention. This only becomes testable when the driver's keys act through the system's belief rather than directly on the car.

The same vignette says when input should pass through more directly: when the driver's inputs are poorly predicted, the car should "temporarily propagate inputs more directly" (the H-metaphor's tightening of the reins). Here this is a precision-weighted fusion (Eq. 10), not a rule.

Authority check (G6)

Held right key from the left lane centre, no noise or drift. When the belief agrees with the driver, mediated steering must give at least the naive condition's authority, so that the conditions differ in what the system infers, not in raw gain.

Generative model

Hidden state: the driver's intended lateral position g ∈ [−144, 144] px (road-relative). Observations: car position x and velocity v (precise), column input u (driver plus noise), road geometry and a tracked obstacle. Each term below is an energy (negative log probability); Eq. 6 adds them up.

Eq. 1 Preference prior, informed by the sensed lanes and relearned (Eq. 12)
$$p_\text{pref}(g) = (1-\varepsilon)\,\frac{c(g)}{\sum c} + \frac{\varepsilon}{N},\qquad c_0(g) = A\,\frac{\sum_k \mathcal N(g;\,L_k,\sigma_L^2)}{Z}$$
Eq. 2 Road boundary (sensed markings, never learned)
$$V_\text{road}(g) = \Big(\tfrac{\max(0,\,|g|-b)}{s_b}\Big)^2$$
Eq. 3 Obstacle hazard: collision footprint, weighted by proximity
$$V_\text{haz}(g) = \pi_\text{obs}\;\sigma\!\Big(\tfrac{w_c-|g-o|}{s_h}\Big),\qquad \pi_\text{obs} = \operatorname{clamp}\!\big(1-\tfrac{d_y}{R},0,1\big)$$
Eq. 4 User model: how a driver with goal g steers
$$\hat u(g;\,x,v) = \tanh\!\big(k\,(g - x - \tau v)\big)$$
Eq. 5 Input precision: a press is strong evidence, no input is weak evidence
$$\Pi_u(u) = \begin{cases}\Pi_u & |u| > 0.5\\ f\,\Pi_u & \text{otherwise}\end{cases}$$
Eq. 6 Free-energy rate at each candidate goal
$$F(g) = \gamma_p\big[-\ln p_\text{pref}(g) + V_\text{road}(g)\big] + \gamma_h V_\text{haz}(g) + \gamma_e\,\tfrac12\,\Pi_u(u)\,\big(u-\hat u(g)\big)^2$$

Why no input is weak evidence (Eq. 5): a driver who is not pressing may simply be content within some tolerance. Treating silence as precise evidence that g = x would let the belief follow the car, which the belief itself moves: a self-confirming loop.

Explore the landscape

The terms of Eq. 6 for one observation. The shaded curve is the belief this observation alone would produce after 1 s, exp(−F·1 s), starting from a flat belief.

How the grid filter works

The belief is a probability distribution over a continuous quantity, the goal g. The filter represents it by its values at N = 145 points, 2 px apart from −144 to 144 px: a vector q whose entries sum to 1. This is a grid-based (point-mass) Bayesian filter (Bucy & Senne, 1971; Arulampalam et al., 2002). Every tick (Δt = 1/60 s) it applies Bayes' rule in two steps, like a Kalman filter but without assuming the belief is Gaussian.

Step 1 · Eq. 7 Predict: the goal may have moved

Chapman–Kolmogorov: the new belief is the old one pushed through the goal's dynamics, \(q^-(g) = \sum_{g'} p(g \mid g')\,q(g')\). The model says the driver's goal performs a random walk, \(p(g \mid g') = \mathcal N(g;\,g',\sigma_D^2\Delta t)\), so this step is a Gaussian blur of the belief (σD = 40 px/√s, about 5 px per tick). The optional term ρ lets the goal jump to a preferred position; it is 0 in the experiment.

$$q^-(g) = (1-\rho\Delta t)\,\big[\mathcal N(0,\sigma_D^2\Delta t) * q_{t-\Delta t}\big](g) + \rho\Delta t\;p_\text{pref}(g)$$
Step 2 · Eq. 8 Correct: weigh every candidate goal by how well it explains this tick

Bayes: \(q_t(g) \propto p(o_t \mid g)\,q^-(g)\). The log-likelihood of one tick is \(-\Delta t\,F(g)\), with F the free-energy rate of Eq. 6. Evidence arrives continuously, so each tick carries Δt seconds' worth of it, and the belief after one second does not depend on the frame rate. The preference prior sits inside F too: it pulls the belief towards preferred positions at rate γp, while step 1 keeps spreading it. Their balance sets the resting width of the belief (about 26 px).

$$q_t(g) \propto q^-(g)\,\exp\!\big(-\Delta t\,F(g)\big)$$
Step 3 · Eq. 9 Normalise and summarise for action

Dividing by the sum gives a proper distribution (the sum is the evidence p(ot)). Action needs a target and a confidence: μ is the mean of the dominant peak (within ±40 px of the highest point, so two peaks are not averaged into the gap between them), and Σ is the variance of the whole belief.

$$\mu = \mathbb E_q[g \mid \text{dominant mode}],\qquad \Sigma = \operatorname{Var}_q[g]$$

What is approximated. Nothing in the inference itself: no linearisation, no Gaussian assumption, no sampling. The only error is the discretisation, and 2 px is small against the narrowest belief (σ = 8 px at reset) and the car (68 px wide).

Step through the filter

Belief before the step (grey, dashed), after step 1 only (light blue, single-tick steps), evidence of the step exp(−Σ Δt F) (amber, own scale), belief after the step (blue). Black line: car.

With 1-tick steps the change is small by design: one tick is 1/60 s of evidence. The 0.25 s steps show how evidence accumulates; in the "right key" case the mass flows across the barrier between the lanes.

Why an exact filter on a grid

VFE Exact inference is the free-energy minimum
$$F[q] = \mathbb E_q\big[\ln q(g) - \ln p(o, g)\big]$$ $$\phantom{F[q]} = D_\text{KL}\big[q(g)\,\|\,p(g\mid o)\big] - \ln p(o) \;\ge\; -\ln p(o)$$

The bound is tight, and F equals surprise, exactly when q is the posterior. The grid filter computes that posterior, so for this model AIF perception and Bayesian filtering coincide (Friston, 2010). Murray-Smith et al. (2025, App. B.1) describe AIF perception as this predict–correct loop and leave the inference method open: variational, exact or Monte Carlo.

The usual continuous-AIF scheme, and the one written in the outline, is the Laplace approximation (Buckley et al., 2017): restrict q to a Gaussian, and let its mean descend the free energy.

Laplace Point belief descending the same landscape
$$\dot\mu = -\kappa\,\frac{\partial F}{\partial \mu},\qquad \Sigma^{-1} = \frac{\partial^2 F}{\partial \mu^2}$$
Grid filterLaplace
Belief shapeany, including two peaksone Gaussian
Approximation2 px discretisation onlyGaussian form, gradient steps
Cost per tick145 points × a 17-point blura few evaluations of F
Scales to1–2 hidden dimensions (Nᵈ points)many dimensions

Why the grid fits here. The hidden state is one-dimensional, so the grid is cheap, and the posterior is often two-peaked (two lanes, or both sides of an obstacle). With the car centred on an obstacle, the Laplace belief sits on the hazard's peak, where the gradient is zero, and stays there. The grid posterior moves its mass to the other lane. Both schemes pass every other scenario in the suite.

Obstacle in the own lane, passive driver

Grid filter: belief q(g) over time (blue), car x (black), μ (dashed)
Laplace: point belief μ ± σ (blue band), car x (black)

Action: precision-weighted pass-through

The assist makes two predictions of how fast the car should move: towards the believed goal, and what the driver's key commands directly. It fuses them by their precisions, so the driver's input passes through more when the belief is uncertain (G4).

Eq. 10 Fused velocity prediction
$$v_\text{goal} = \lambda(\mu - x),\qquad v_\text{dir} = V_u\,u,\qquad v^* = (1-\alpha)\,v_\text{goal} + \alpha\,v_\text{dir}$$ $$\alpha = \frac{\Pi_d}{\Pi_\text{goal} + \Pi_d},\qquad \Pi_\text{goal} = \Sigma^{-1}\,e^{-\beta\,\bar e^2}$$ $$\tau_s\,\dot{\bar e^2} = \big(u-\hat u(\mu;\,x,v)\big)^2 - \bar e^2$$
Eq. 11 Reflex arc on the velocity prediction error
$$F_\text{assist} = \operatorname{clamp}\!\big(K\,(v^* - v) + c\,v_\text{chassis},\;\pm F_\text{max}\big)$$

Action follows the AIF reflex-arc account: the force reduces the prediction error on the car's velocity. The term c·vchassis compensates the plant's own damping, which is part of the system's forward model. v is road-relative, because the lane markings give the road's motion.

Two readings of "reins". With β = 0 (the experiment's setting), α depends only on the belief's spread. A pressed key is precise evidence, so α falls while the driver presses and rises when they let go. With β > 0, α also rises while the driver's inputs are poorly predicted by the believed goal, e.g. when they steer against it, which is closer to Murray-Smith's wording. With β = 3 the suite still passes, except the passive-driver obstacle case.

Reins during a lane change

Car x, belief μ ± σ, key press (shaded)
Pass-through α (0 = goal-directed only, 1 = driver input only)
α as a function of belief spread σ (Eq. 10, β = 0)

Acceptance scenarios

Each scenario tests one or more design goals (Design goals slide). Criteria apply to the new controller; the others are shown for reference, as meeting or missing the same criterion. Click a row for its trajectories and the belief over time. Headless: node demos/aif_proposal/run_scenarios.mjs.

Select a scenario

Car position x(t), every controller in the scenario
New controller: belief q(g) over time (blue), car x (black), μ (dashed)

Learning the preference prior

Eq. 12 Dirichlet-style counts absorb the posterior
$$c(g) \leftarrow c(g) + \eta\,\Delta t\; q_t(g)$$

The prior starts from the sensed lane geometry, with mass A (Eq. 1). Every second, η of probability mass is added where the driver's goal was believed to be. A driver who persistently drives somewhere else reshapes the prior, until the assist holds that position by itself (G5).

This is preference learning in the sense of Murray-Smith et al. (2025, §3.5.2): preference priors learned from interaction rather than fixed in advance. A and η set the timescale. Because intentions are never enumerated, a centre preference is just as representable as a lane.

In the experiment learning is off (η = 0) and the prior is reset every trial, so conditions stay stationary and every trial replays from its own log. The playground and the sandbox turn it on.

Preference prior before (dashed) and after learning
Afterwards: released at that position with no input for 15 s

Sandbox

Click the road, then steer with A/D or ←/→ (use the ‹ › buttons to change slides).
Belief over goal g: prior (violet), initial prior (light), input likelihood (amber), hazard (red), posterior (blue). Lines: car x (black), μ (blue), autopilot goal (green).
Last 10 s: belief q(g), car x (black), μ (dashed)

Previous vs new, side by side

PreviousNew
Hidden statelane log-odds (two lanes)intended position g, continuous
Intentionshard-coded lanes, fixed baseline lanepreference prior from sensed geometry, learnable
Evidenceleaky integral of the column inputlikelihood from a user model, input-dependent precision
Inferenceheuristic updateexact Bayesian filter on a 2 px grid
After a lane changedecays back to the start lanestays: the new lane is as preferred as the old
Obstaclepushes log-odds; a passive driver collidescollision-footprint hazard on g; a passive driver avoids
Driver authorityfixed blend into the targetprecision-weighted pass-through, authority ≥ naive
Forceproportional pull to a targetreflex arc on velocity prediction error
Auditabilityplan and code had divergedequation-tagged code, tested scenarios, exact replay

The experiment plan and preregistration describe the new model (control-mode section, conflict and effort definitions).

References

  • Arulampalam, M. S., Maskell, S., Gordon, N., & Clapp, T. (2002). A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 50(2), 174–188.
  • Buckley, C. L., Kim, C. S., McGregor, S., & Seth, A. K. (2017). The free energy principle for action and perception: A mathematical review. Journal of Mathematical Psychology, 81, 55–79.
  • Bucy, R. S., & Senne, K. D. (1971). Digital synthesis of non-linear filters. Automatica, 7(3), 287–298.
  • Feldman, H., & Friston, K. J. (2010). Attention, uncertainty, and free-energy. Frontiers in Human Neuroscience, 4, 215.
  • Flemisch, F. O., Adams, C. A., Conway, S. R., Goodrich, K. H., Palmer, M. T., & Schutte, P. C. (2003). The H-Metaphor as a guideline for vehicle automation and interaction (NASA/TM-2003-212672).
  • Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11, 127–138.
  • Haggard, P., Clark, S., & Kalogeras, J. (2002). Voluntary action and conscious awareness. Nature Neuroscience, 5, 382–385.
  • Lanillos, P., & Cheng, G. (2018). Adaptive robot body learning and estimation through predictive coding. IROS 2018.
  • Mecacci, G., & Santoni de Sio, F. (2019). Meaningful human control as reason-responsiveness: The case of dual-mode vehicles. Ethics and Information Technology.
  • Murray-Smith, R., et al. (2025). Active Inference and Human–Computer Interaction. ACM Transactions on Computer-Human Interaction, 32(6), Article 63.
  • Santoni de Sio, F., & van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philosophical account. Frontiers in Robotics and AI, 5, 15.
  • Tapal, A., Oren, E., Dar, R., & Eitam, B. (2017). The Sense of Agency Scale. Frontiers in Psychology, 8, 1552.
  • Wen, W., Yamashita, A., & Asama, H. (2015). The sense of agency during continuous action: Performance is more important than action-feedback association. PLoS ONE, 10(4), e0125226.
  • Tiebosch, J. P. (2025). PhD outline (User Schema; Phases 1–3).

Extension · prototype

Learning the steering-column disturbance

The slides so far describe the controller the experiment runs. The next two show a possible extension: a prototype that is not part of the experiment's controller. It lives in demos/aif_proposal/extensions/disturbance_learning.js and builds on the experiment's controller without changing it.

Question

In lane keeping the car wobbles by about 3 px. Why does learning the preference prior not reduce it?

Idea

Add a hidden disturbance to the generative model, and learn its statistics from prediction errors (precision learning).

Result

The noise-driven wobble falls by about three quarters within a minute; every scenario still passes.

Extension · prototype · not used by the experiment

A hidden disturbance in the generative model

The base model treats the column input as the driver's input plus white noise. The real disturbance is smooth: it pushes in the same direction for about a second at a time, so it keeps nudging the belief (as weak evidence, Eq. 5) and passes through the direct path (Eq. 10).

Learning the preference prior (Eq. 12) cannot remove this. The prior describes where the driver wants to be, not what disturbs the column, and its well only narrows until it matches the belief's own width.

The extension adds a hidden disturbance d to the model and learns its variance and timescale from the input. In AIF terms this is precision learning: precisions are themselves inferred from the size of prediction errors (Feldman & Friston, 2010).

E1–E5 Disturbance model, its belief, and learning
$$\text{E1}\quad d_t = a\,d_{t-\Delta t} + w_t,\qquad a = e^{-\Delta t/\tau_d},\qquad \operatorname{Var}[d] = s_d^2$$ $$\text{E2}\quad u = u_\text{driver} + d$$ $$\text{E3}\quad K = \frac{P}{P+\sigma_r^2},\qquad \hat d \leftarrow \hat d + K\,(u - \hat d)\qquad\text{(no key pressed)}$$ $$\text{E4}\quad m_2 \leftarrow m_2 + \tfrac{\Delta t}{\tau_L}\,(u^2 - m_2),\qquad s_d^2 = \max(0,\,m_2-\sigma_r^2)$$ $$\phantom{\text{E4}}\quad \rho = \frac{\mathbb E[u_t\,u_{t-\Delta t}]}{m_2},\qquad \tau_d = \frac{-\Delta t}{\ln \rho}$$ $$\text{E5}\quad u' = u - \hat d \;\;\text{goes into the base model (Eq. 4–10)}$$

Where the wobble comes from (live)

Car spread over 20 s of lane keeping with no input, for the experiment's controller and for the extension after 2 min of learning.

running…

Assumptions and limits

  • Mean-field: q(g, d) ≈ q(g) q(d). The disturbance is estimated from the raw input, the goal from the corrected input u′.
  • Only while no key is pressed is d observed (E3); during a press it is only predicted forward (E1), because the driver's contribution is then unknown and large.
  • σr says the driver's own contribution is small when no key is pressed. That holds for a keyboard. With an analogue wheel, small genuine corrections would partly be explained away as disturbance.
  • A simplified disturbance: the real one is a sum of sinusoids. The learned τd reflects its smoothness, not its period.
  • Not for the experiment: the AIF condition would keep learning during a block (non-stationary), and only that condition would get a noise model.

Extension · prototype · not used by the experiment

Learning curve

Car spread in successive 15 s windows of passive lane keeping: experiment's controller (grey) and extension (blue)
Last 5 s: column input u (amber) and the learned disturbance estimate d̂ (blue)

Scenarios with a trained disturbance model

The acceptance scenarios (except the learning and replay ones), each started with a model that first learned for 2 min. Authority, filtering and obstacle behaviour must be unchanged.

press Run