Terminology

Nomina si nescis, perit et cognitio rerum.
(If you know not the names of things, the knowledge of things themselves perishes.)

Carl von Linné

Active inference framework

Notation notes

• Time sequences: Time sequences are indexed with the subscript \(k\) where the subscript \(0\) represents the present time. Negative numbers represent the past and positive numbers represent the future. The time intervals between the time samples are not assumed to be equidistant.
• Shorthand: A random variable without a subscript, e.g., \(s\), is a shorthand for either a a random variable at a certain point in time or a time sequence of the random variable, as given by the context. When explicitly referring to a random variable at a specific time sample, a single subscript is used, e.g., \(s_0\).
• Tilde: A tilde over a probability distribution, like \(\tilde p(s)\) or \(\tilde p(o)\), means that the distribution is a normative distribution, rather than a neutral prediction. It defines what states the system ought to be in rather than which states it is in.
• Structure: While the goal has been to order the definitions from atomary and primitive terms to composite terms, this has not always succeeded. Some scrolling both up and down may be needed to fully understand a term.

System: An arrangement of interacting elements that together realize functions. Examples include information systems, biological organisms, ant hills, medical devices, and space stations.

Open system: A system that has external interactions. Such interactions can take the form of information, energy, or material transfers into or out of the system boundary.

Distribution, probability distribution: A mathematical function or table that describes the likelihood of all possible outcomes for a random variable, discrete or continuous. Quantified with a probability density function (continuous random variables) or a probability mass function (discrete random variables).

State vector: An ordered array of state variables that mathematically defines a single point within a state space \(\Omega\). Within active inference, a state vector represents either a controlled-system state or an information state within a controller.

State variable: A component of a state vector. State variables can be categorical or continuous. Controllers in contact with or close to the controlled system work with continous state variables such as joint angle and body temperature. Higher-level controllers work with more abstract categorical state variables such as “mug of coffee in hand” or “social status”.

State: The current value of the state vector.

State space, \(\Omega\): The bounded, multi-dimensional mathematical space (or manifold) comprising all permissible state vectors for a given controller or controlled system. Its dimensions are spanned by the domains of its constituent state variables. If a state vector consists of \(N\) distinct state variables, where each variable \(i\) belongs to its own domain \(\mathcal{X}_i\) (which can be continuous, such as \(\mathbb{R}\), or categorical), the complete state space is the Cartesian product of these domains:

$$\Omega = \mathcal{X}_1 \times \mathcal{X}_2 \times \dots \times \mathcal{X}_N = \prod_{i=1}^N \mathcal{X}_i$$

Any specific state vector, e.g., a representational state \(s\) or a physical state \(\eta\), is a single point within this coordinate system, uniquely defined by the simultaneous values of its constituent state variables. For a representational state we have:

$$s = \begin{bmatrix} s^{(1)} \\ s^{(2)} \\ \vdots \\ s^{(N)} \end{bmatrix} \in \Omega_s$$

Physical state, \(\eta\): The subvector of the controlled-system state comprising those state variables that describe the system’s physical configuration, such as mechanical, chemical, thermodynamic, or other directly physically instantiated quantities. If \(x\) denotes the full controlled-system state, then:

$$\eta = P_{\eta} x$$

where \(P_{\eta}\)​ selects the physical state variables from the full controlled-system state vector.

The physical state of a passive (uncontrolled) physical system evolves according to the equation:

$$d\eta = f(\eta)dt + d\omega_\eta$$

where \(\omega_\eta\) represents random perturbations of the state.

Examples of state variables of the physical state include the physical temperature of the human body, the ATP concentration in a cell, and the momentum of a vehicle.

Generalized coordinates of motion: Representational state vector representing a dynamic process, e.g., motion. Its state variables represent the quantity, the time derivative of the quantity, the second time derivative of the quantity etc. For motion in the x-direction the generalized coordinates of motion would for instance be:

$$s = \begin{bmatrix} x \\ \dot x \\ \ddot x \\ \vdots \end{bmatrix}$$

Where \(x\) is the current position of the object in x-direction, \(\dot x = dx / dt\) etc.

Macrostate, \(z, s\): A higher-level state variable that summarizes, coarse-grains, or otherwise represents a configuration of lower-level states. A macrostate may occur as a property of a controlled system or as a representational state within a controller. Whether a state is a macrostate concerns its level of description, not its location or role in the control hierarchy. An example of a macrostate is \(\text{social status}\).

If \(x\) denotes the full controlled-system state, then:

$$z = P_z x$$

where \(P_z\)​ selects the macrostate variables from the full controlled-system state vector.

Controlled-system state, \(x, s\): The state of either a controlled physical system such as a body or the state of a target controller being regulated by a source controller (in which case the target controller is the controlled system). The controlled-system state consists of the directly controlled physical state variables and pertinent macrostates.

A controlled-system state is a “God’s eye” vector that can only be estimated through observations and is therefore never fully accessible to a controller.

Examples of state variables of the controlled-system state include \(\text{body temperature}\), \(\text{mug of coffee in hand}\), and \(\text{social status}\).

Generative process: The physical dynamics of a controlled system. It is the process that the controller’s predictive generative model attempts to simulate. It is defined by the continuous time evolution of the physical state \(\eta\):

$$d\eta = f(\eta, \eta_a)dt + d\omega_\eta$$

The evolution of macrostates \(z\) can be described as discrete time process:

$$p(z_k \mid z_{k-1}, u_{k-1})$$

where \(u\) is a macroaction consisting of a trajectory of action states.

Controlled system: A system that is kept within a limited, defined set of states by the actions of a controller. Examples of controlled physical systems include individual cells, organisms, tissue and organs, the whole human body, and the habitat of organisms capable of modifying their habitat.

A target controller is a controlled system from the perspective of a source controller.

Control: The act of keeping a controlled system within a limited, defined set of states.

Predictive control: Control informed by a dynamic system model that predicts future controlled system states over a time horizon.

Active inference: A general form of predictive control encompassing perception, action, planning, decision-making and learning, employed by biological organisms to keep them in a non-equilibrium steade state, i.e., keep their controlled-system state on the viable manifold.

Non-equilibrium steady state (NESS): A physical state of an open system that is constantly kept out of equilibrium by external forces, such as energy, matter, or information flux. Unlike equilibrium, NESS features persistent currents (e.g., heat flow, particle transport), continuous entropy production, and is maintained through constant interaction with its environment.

Physical viable manifold, \(\mathcal{M}_{\eta}\): The set of physical states that a organism in NESS can occupy.

Viable manifold, \(\mathcal{M}_{x}\): The set of controlled-system states that are consistent with the persistency of NESS.

Organism: A biological system maintaining NESS (as long as it is alive).

Controller, \(\mathcal{C}_i\): A system that observes and acts upon a controlled system to keep it within a limited, defined set of states close to the target controlled system state \(\tilde x\), the viable manifold \(\mathcal{M}_{x}\).

Closed-loop controller: A controller that utilizes a continuous stream of observations \(o\) to update its recognition distribution \(q(s)\) and to generate actions \(a\) to keep a controlled system on its viable manifold.

Open-loop controller: A controller that executes actions \(a\) without updating its recognition distribution \(q(s \mid \theta)\) from real-time observations \(o\). Through learning, much of the computational burden has been amortized into a policy-conditioned generative dynamical model:

$$\dot \mu = f(\mu, \pi)$$

where \(\pi\) is the chosen policy, in this case basically the decision to do something like striking a golf ball from tee.

In active inference, very fast actions must be open-loop because of observation-error calculation-action latency.

Controller network, \(\mathcal{C}\): A network of interacting controllers separated by Markov blankets, collectively controlling an organism.

Source controller, \(\mathcal{C}_s\): A controller partially controlling a target controller by modifying the setpoint distribution of the target controller.

Target controller, \(\mathcal{C}_t\): A controller receiving empirical setpoint distributions from one or more source controllers. A target controller combines all received empirical setpoint distributions with its own structural setpoint distribution to a total (combined) setpoint distribution that it then pursues:

$$\tilde p_{total}(s) \propto \tilde p_{struct}(s) \prod_{i=1}^I \tilde p_{emp, i}(s)$$

Terminal controller: A target controller at the absolute boundary of the controller network \(\mathcal{C}\) that has no subordinate target controllers. It is the only type of controller whose action \(a\) is translated directly by an effector across the physical Markov blanket into a continuous physical action state \(\eta_a\). It constitutes the strict mathematical barrier where the informational domain (active inference) ends and the unguided physical domain (thermodynamics and mechanics) begins.

Reflexive controller: A controller that operates strictly in the current moment. It receives a real-time stream of observations. From the observation in any given moment it infers a recognition distribution and an action to minimize (variational) free energy using its generative normative model, also in (near) real-time. A reflexive controller doesn’t have any memory and it doesn’t plan ahead. While it does not have memory of past observations, it internally represents the history of the state via the generalized coordinates of motion estimated in the current moment.

Temporal controller: A controller that operates over the time sequence \(-N:K\):

• It infers its recognition distribution \(q(s_{-N:0}, \pi_{-N:-1})\) from past observations \(o_{-N:0}\), using the predictive generative model, minimizing retrospective free energy. This process is sometimes called smoothing. Weighing in past observations renders a more accurate recognition distribution than can be inferred by a reflexive controller.
• It infers a waypoint distribution guiding future actions by generating alternative policies from its policy space \(\mathcal{U}\), using the predictive generative model, selecting a policy that minimizes expected free energy \(G(\pi)\), which combines preference-related and epistemic consequences of predicted future observations. The temporal controller thus looks past the immediate free energy gradient to optimize free energy over a planning horizon.

Target controlled-system state distribution, \(\tilde p(x)\): The objective, normative probability distribution over the controlled-system state space \(\Omega_x\) that the controlled system must occupy to maintain its non-equilibrium steady state (NESS) and structural integrity. It is the controlled-system counterpart to the controller’s internal setpoint distribution \(\tilde p(s)\). While \(\tilde p(s)\) is a parameterized data structure evaluated inside the controller, \(\tilde p(x)\) is an objective property of the physical system’s morphology. A perfectly calibrated controller is one where the internal setpoint distribution \(\tilde p(s)\) accurately maps to the target controlled-system state distribution \(\tilde p(x)\) via the observation model.

The viable manifold \(\mathcal{M}_x\) defines the safety envelope wherein the system maintains its NESS over time. It is defined by the target controlled-system state distribution:

$$ \mathcal{M}_x = { x \in \Omega_x : \tilde p(x) > \epsilon_x } $$

Where:

• \(x\): The controlled-system state vector.
• \(\Omega_x\): The controlled-system state space.
• \(\tilde p(x)\): The target controlled-system state distribution.
• \(\epsilon_x\): The critical viability threshold (the probability boundary below which the physical structure degrades irreversibly).

Target controlled-system state, \(\tilde x\): The mode of the target controlled-system state distribution \(\tilde p(x)\). This corresponds precisely to the physical setpoint or reference state in classical control theory.

Information state, \(s, o, a\): A state vector whose state variables (components) are random variables representing quantities of interest within a controller. The three types of information states in active inference are:

• Observation, \(o\)
• Action, \(a\)
• Representational state, \(s\)

Out of these, the action and the observation are point estimates. The controller infers an implicit precision of the observation which is used the observation model to modulate perceptual updating (an observation in a snowstorm has low precision). Because observations aren’t perfect, there will always be uncertainty in the estimation of the current representational state, the recognition state. The recognition state is therefore represented as a probability distribution \(q(s \mid \theta)\) rather than a point estimate.

Representational state, \(s\): The random variable that spans the controller’s internal state space \(\Omega_s\). It serves as the mathematical proxy for the physical state \(\eta\) of the controlled system within the generative model. It acts as the continuous or categorical domain (support) over which the controller evaluates distributions, such as the recognition distribution \(q(s \mid \theta)\) and the setpoint distribution \(\tilde p(s)\).

Representational state variables vary in their level of abstraction:

Low-level representational state variables represent immediate, localized physical states ($\eta$) of the controlled system alongside their temporal derivatives up to a given order (typically order 2, capturing velocity and acceleration), and in hybrid engineering applications, their temporal integrals. This allows a reflexive controller to internally represent immediate physical trajectories and accumulated state histories using an augmented set of generalized coordinates of motion.

High-level representational state variables represent compressed, complex, long-term physical states and controlled-system macrostates such as \(\text{mug of coffee in hand}\), \(\text{social status}\), and \(\text{destination of flight}\).

Surprise, \(– \ln \tilde p(o)\): The negative logarithm of the probability of an observation as given by the generative model. The viable manifold of the controlled system can in a well calibrated controller be described as a viable manifold over observations:

$$\mathcal{M}_{x} \approx \{ o=g(x) \in \Omega_o : \tilde p(o) > \epsilon_o \}$$

In contrast to the arbitrary, human-engineered cost functions used in classical predictive control, surprise serves as an ontologically grounded loss function. It bounds the probability that the controlled-system state \(x\) is leaving the physical viable manifold \(\mathcal{M}_x\). Thus, active inference treats sensory deviations not merely as tracking errors, but as direct thermodynamic deviations from NESS.

Recognition state, $\hat s$: The controller’s single point-estimate of the controlled system’s state. In a deterministic controller, it is the direct output of the observer or state estimator based on current and past observations. In a probabilistic controller, it corresponds to the mode (most probable value) of the recognition distribution:

$$\hat s = \underset{s}{\arg\max} \, q(s \mid \theta)$$

Recognition distribution, \(q(s \mid \theta)\), \(q(s, \pi \mid \theta)\):

\(q(s \mid \theta)\) is a variational distribution that acts as a tractable approximation of the intractable posterior distribution \(p(s \mid o)\) in a reflexive controller. It is the best estimate of the current representational state of the controller.

\(q(s, \pi \mid \theta)\) is a variational distribution that acts as a tractable approximation of the intractable posterior distribution \(p(s, \pi \mid o) = p(s \mid \pi, o)p(\pi \mid o)\) in a temporal controller. \(s\) and \(\pi\) are past sequences of the respective random variables.

\(\theta\) represents the recognition distribution parameters such as the mean \(\mu\) and the precision \(\Pi\).

Recognition precision, \(\Pi\): The precision of the recognition distribution.

Observable state, \(\eta_o, s_o\): \(\eta_o\) is a vector of a subset of the physical state variables that interact with the controller’s transducers. Within a network of controllers, the observable state of a target controller \(s_{o, t}\) is the full set of parameters \(\theta_t\) defining its recognition distribution \(q(s_t \mid \theta_t)\) (e.g., mode \(\hat s_t\) and precision \(\Pi_t\)).

Observation, \(o, o_{[k_1:k_2]}\): A sequence of point-estimate information states. At the physical boundary, it is derived via transducers from \(\eta_o\). Within a controller network, the observation \(o_s\) received by a source controller strictly equals the recognition state (mode \(\hat s\)) of the target controller’s observable state \(s_{o, t}\). Secondary parameters from \(s_{o, t}\), such as precision \(\Pi_t\), bypass the observation vector to directly parameterize the variance of the source controller’s observation model \(p(o_s \mid s_s; \Pi_t)\).

Action state, \(\eta_a, s_a\): In a physical system, a continuous physical state (\(\eta_a\)) generated by an effector with the intention to force or maintain the system’s physical state, counteracting environmental dynamics. It is the physical realization of the action \(a\), translated across the physical Markov blanket into a continuous force or flux:

$$d\eta = f(\eta, \eta_a)dt + d\omega_\eta$$

Within a controller network, the action state of a target controller \(\mathcal{C}_t\) from the perspective of a source controller \(\mathcal{C}_s\) is the setpoint distribution \(\tilde p_t(s_t)\) generated by the \(\mathcal{C}_s\) action \(a_s\).

Macroaction, \(u\): A coarse-grained, usually temporally extended change or activity of the controlled system that is realized through a trajectory of physical action states and may influence one or more macrostates. A macroaction is described at a higher level than its constituent physical actions and need not correspond one-to-one to any decision or information state within the controller.

Action, \(a\): A continous information state that drives an action state in the controlled system via an effector. The action state is a function of the action:

$$\eta_a = h(a)$$

Markov blanket \(b\): The set of boundary states \(\eta_b = \{\eta_o, \eta_a\}\), or \(s_{b,t} = \{s_{o,t}, s_{a, t}\}\) if separating a source controller \(\mathcal{C}_s\) and a target controller \(\mathcal{C}_t\) (the Markov blanket is defined in terms of the target states).

The Markov blanket mediates all information between the controller states and controlled system states.

Mathematically the Markov property implies that:

$$p(s, \eta | \eta_b) = p(s | \eta_b)p(\eta | \eta_b)$$

or, if the Markov blanket separates a source controller \(\mathcal{C}_s\) and a target controller \(\mathcal{C}_t\):

$$p(s_s, s_t \mid s_{b, t}) = p(s_s \mid s_{b, t})p(s_t \mid s_{b, t})$$

We will here assume that the Markov blanket is defined at the cellular level. In principle, coarser and finer descriptions are possible.

Agency model (\(\frac{\partial o}{\partial a}\)): The controller’s internal mapping that defines its capacity to alter the controlled system state and therefore its observations (\(o\)) by emitting an action (\(a\)) to drive effectors across the Markov blanket. It serves as the mathematical gatekeeper determining whether a free energy gradient is resolved via action (when agency \(\neq 0\)) or forced into perception/acceptance (when agency \(= 0\)).

Observation model, \(p(o \mid s)\): The controller’s mapping between representational states \(s\) and expected observations \(o\). It defines how the controller expects a given physical state represented by \(s\) to manifest as sensory data (transformed by transducers).

Its fidelity can be characterized by its sensitivity (the probability of an observation correctly indicating the presence of a state) and specificity (the probability of an observation correctly indicating the absence of a state). These parameters determine the precision of the likelihood and dictate how much the recognition distribution \(q(s \mid \theta)\) will shift its parameters upon receiving a new observation.

While phylogenetically constrained by the physical properties of the transducers, its specific parameters \(\phi\) are continuously but slowly updated via the learning update rate \(K_\phi\) to account for morphological changes and environmental calibration. The observation model can be assumed to be constant during waypoint inference though.

Transition model, \(p(s_k \mid s_{k-1}, \pi_{k-1})\): The “physics engine” of a temporal controller. It dictates how the controller expects the representational state \(s_{k-1}\) to evolve into the next state \(s_k\) as a consequence of a decision \(\pi_{k-1}\).

Deontic model, \(\tilde p(\pi_{k-1} \mid s_{k-1})\): Dictates the controller’s prior beliefs regarding which decisions are afforded (believed to be physically executable), historically habitual, or mandated by heuristics (duties) when in state \(s_{k-1}\), evaluated prior to any computation of prospective consequences.

Generative dynamical model, \(\dot \mu = f(\mu, \pi; \phi)\): A learned mapping parameterized by the generative model parameters \(\phi\) that guides a ballistic or semi-ballistic movement such as a golf swing. This model allows a controller to execute goal-directed physical changes when the real-time observation model \(p(o \mid s)\) is partially or fully inactive,e.g., due to timing constraints, bypassing the mathematical requirement for continuous free energy gradient descent.

Amortized perception model, \(\theta(o; \phi)\): A learned mapping parameterized by the generative model parameters \(\phi\) that directly translates an observation \(o\) into the parameters \(\theta\) of the recognition distribution \(q(s \mid \theta)\). It acts as a fast inverse model (e.g., \(\theta(o; \phi)\)) that instantly outputs a state estimate, such as the recognition state \(\hat s\), based on current sensory input. This model allows a controller to perform a rapid feedforward sweep to immediately minimize instantaneous free energy \(\mathcal{F}_{inst}\), bypassing the computational latency of iterative gradient descent governed by the perception update rate \(K_\theta\).

In biological implementations of active inference (predictive coding), this model is physically realized by the optimized synaptic weights of feedforward connections, which drive the recognition state \(\hat s\) to the free energy minimum in a single bottom-up sweep.

Update rate, \(K_\theta, K_a, K_\phi\): The rate constant matrices (with dimension \([\text{time}]^{-1}\)) that govern the velocity of free energy gradient descent. They act as the physiological gain translating computational prediction errors into parameter updates or physical forces or fluxes.

• Perception update rate, \(K_\theta\): Dictates the rate of change of the recognition distribution parameters \(\theta\).
• Action update rate, \(K_a\): Dictates the rate of change of the action state \(a\).
• Learning update rate, \(K_\phi\): Dictates the rate of change of the generative model parameters. Typically much smaller than \(K_\theta\) and \(K_a\).

Setpoint state, $\tilde s$: A specific target point in the state space (\(\tilde s \in \Omega_s\)) that the controller is configured to achieve and maintain. It serves as the functional reference value toward which the controller’s physical actions or internal perceptual biases are directed. In a probabilistic controller, it corresponds to the mode of the setpoint distribution:

$$\tilde s = \underset{s}{\arg\max} \, \tilde p(s)$$

Note that for Gaussian distributions the mode equals the mean. Categorical distributions don’t have a well-defined mean but they have a mode (a most probable category).

Setpoint distribution, \(\tilde p(s), \tilde p(o)\): A probability distribution over representational states or observations that defines the target regions the controller is biased to occupy (the viable manifold). Mathematically, it occupies the position of a Bayesian prior. Functionally, it fulfills two simultaneous roles depending on the gating of the update rates:

• Pragmatic target (action): The setpoint distribution drives action to shift the controlled system’s state toward the the physical state represented by the setpoint state \(\tilde{s}\) (which is the mode of \(\tilde p(s)\)).
• Epistemic bias (perception): The setpoint distribution acts as a continuous perceptual expectation, pulling the recognition distribution \(q(s \mid \theta)\) toward it. The strength of this bias is inversely proportional to the precision of the observation model \(p(o \mid s)\). When action is structurally impossible, resolving the free energy gradient relies entirely on this epistemic bias.

The total setpoint distribution steering the controller is the integrated product of two components:

1. Structural setpoint distribution, \(\tilde p_{struct}(s)\): Hardcoded, phylogenetically or architecturally endowed probability distribution with high probabilities allocated to preferred representational states on the viable manifold (e.g., a distribution with a sharp peak at the optimal human body temperature of 37°C).
2. Empirical setpoint distribution, \(\tilde p_{emp}(s)\): Probability distribution dynamically generated by a higher-level source controller and passed down to serve as a pragmatic target or epistemic bias in the target controller. When passed from a temporal controller calculating in discrete time steps to a continuously operating reflexive controller, the parameters of this distribution (such as the setpoint state \(\tilde s\)) are held constant as continuous reference values between discrete update intervals.

Setpoint precision, \(\tilde \Pi\): The precision of the setpoint distribution.

Setpoint observation, \(\tilde o\): The mode of the setpoint distribution when the support is observations. This is is not often used in active inference.

Prediction error: The mathematical discrepancy between the empirical observation \(o\) and the expected observation, given the current recognition distribution \(q(s \mid \theta)\) and the observation model \(p(o \mid s)\). In a reflexive controller, it corresponds to the gradient of the instantaneous free energy \(\mathcal{F}_{inst}\). It serves as the direct computational signal that, when multiplied by the respective constant update rates (\(K_\theta\) and \(K_a\)), dictates the velocity of belief updating and physical action.

Generative model, \(\tilde p(s, o), p(s, o, \pi)\): Mathematical constructs and shorthands used to describe active inference algorithms. Active inference defines two types of generative models:

• Normative generative models
• Predictive generative models

Normative generative model, \(\tilde p(s, o)\): A mathematical shorthand representing the joint probability distribution over states and observations that the controller is biased to occupy. It does not exist as a single biological or physical structure; rather, it is the computational product of evaluating the controller’s observation model \(p(o \mid s)\) and its setpoint distribution \(\tilde p(s)\) simultaneously. It serves as the theoretical target for free energy minimization:

$$\tilde p(s, o) = p(o \mid s) \tilde p(s)$$

Planning horizon: The future horizon over which a temporal controller plans its free energy minimization. It extends over the subscripts \([1 \ldots K]\), i.e., representational states \(s_1\) to \(s_K\). The planning horizon is defined by the upper index \(K\).

Retrospective horizon: The past horizon over which a temporal controller infers its recognition distribution \(q(s_{-N:0}, \pi_{-N:-1})\). The retrospective horizon is defined by the lower index \(N\).

Predictive generative model, \(p(s, o, \pi)\): A mathematical shorthand representing the simulated joint probability distribution of states, observations, and decisions over time. Like the normative gnerative model, it is not a monolithic physical structure. It is the transient computational output generated when a temporal controller continuously multiplies its distinct internal components—the observation model, transition model, and deontic model—to evaluate counterfactual trajectories or infer past states. It decomposes for one time step as:

$$p(s_k, o_k, \pi_{k-1} \mid s_{k-1}) = p(o_k \mid s_k) p(s_k \mid s_{k-1}, \pi_{k-1}) \tilde p(\pi_{k-1} \mid s_{k-1})$$

The predictive generative model for waypoint inference over the planning horizon is:

$$p(s_{0:K}, o_{1:K}, \pi_{0:K-1}) = p(s_0) \prod_{k=1}^K p(o_k \mid s_k) p(s_k \mid s_{k-1}, \pi_{k-1}) \tilde p(\pi_{k-1} \mid s_{k-1})$$

Where:

• \(s_{0:K}\): Sequence of representational states.
• \(o_{0:K}\): Sequence of observations.
• \(\pi_{0:K-1}\): Sequence of decisions; policy.
• \(p(s_0)\): Current representational state distribution, approximated by \(q(s_0 \mid \theta)\).
• \(\tilde p(s_k \mid s_{k-1}, \pi_{k-1})\): Transition model predicting state evolution based on the previous state and decision.
• \(p(\pi_{k-1} \mid s_{k-1})\): Deontic model defining decision priors.

The predictive generative model for perceptual inference over the retrospective horizon is:

$$p(s_{-N:0}, o_{-N:0}, \pi_{-N:-1}) = p(s_{-N}) \prod_{k=-N}^0 p(o_k \mid s_k) \prod_{k=-N}^{-1} p(s_{k+1} \mid s_{k}, \pi_{k}) \tilde p(\pi_{k} \mid s_{k})$$

with

$$p(s_{-N}) = \mathbb{E}_{q(s_{-N-1}, \pi_{-N-1})}[p(s_{-N} \mid s_{-N-1}, \pi_{-N-1})]$$

Waypoint state, \(s, s_{[1:K]}\): A sequence of intermediate, counterfactual representational states projected into the future to be visited on a trajectory toward a setpoint state. It represents the mode of a waypoint distribution (\(q(s_{[1:K]})\)) derived via waypoint inference.

Waypoint distribution, \(q(s_{[1:K]})\): A distribution of a waypoint states.

Waypoint inference: The process of deriving a waypoint distribution. Referred to as planning in control theory.

Generative model parameters, \(\phi\): The parameters (analogous to weights in deep learning) defining the generative model structure. Learning is defined as the optimization and updating of \(\phi\).

Computational viable manifold, \(\mathcal{M}_s\): The set of information states within which the controller’s internal model can successfully minimize free energy:

$$\mathcal{M}_s = \{ s \in \Omega_s : \mathcal F(s) < \epsilon_s \}$$

where \(\mathcal F(s)\) is free energy and \(\epsilon_s\) is the maximum threshold of prediction error before the control algorithm loses stability.

Decision, \(\pi_k\): Decision, \(\pi_k\): An internal discrete control parameter that conditionally selects a specific state transition mapping within the transition model. It functions in two distinct modes depending on the phase of the control loop:

• Counterfactual (during waypoint inference): A hypothetical operator applied within the predictive generative model to simulate alternative future trajectories of the representational state \(s\).
• Realized (during policy execution): An internal reference signal—derived from the realized policy \(\pi^*\)—that dictates the actual target state trajectory. It achieves this by shifting the empirical setpoint distribution passed to subordinate controllers or amortized action models.

It acts as a parameter altering state transition probabilities:

$$p(s_k \mid s_{k-1}, \pi_{k-1}) = B_{\pi_{k-1}} \Rightarrow q(s_k \mid \pi_{k-1}) = \sum_{s_{k-1}} p(s_k \mid s_{k-1}, \pi_{k-1}) q(s_{k-1})$$

Where:

• \(B_{\pi_{k-1}}\): Transition matrix representing the state transition probabilities from time sample \(k-1\) to \(k\) under decision \(\pi_{k-1}\).
• \(q(s_{k-1})\): Internal distribution over states at the previous time sample.

Note on standard literature: Decision is frequently denoted as “action” in traditional active inference literature, confusingly overloading the term with the physical/informational action state (\(a\)).

Policy, \(\pi\): A sequence of decisions \(\pi = (\pi_0, \pi_1, \dots, \pi_{K-1})\) evaluated within a bounded policy space (\(\mathcal U\)) during waypoint inference to guide the system toward its setpoint distribution.

Policy space, \(\mathcal U\): The bounded set of all possible sequences of decisions available to a controller.

Realized policy, \(\pi^*\): The single sequence of decisions selected from the inferred policy distribution \(q(\pi)\) for actual deployment by the controller. While waypoint inference evaluates a probability distribution over the policy space \(\mathcal{U}\), the controller must collapse this distribution into a single realized policy to orchestrate downstream actions. Under baseline conditions, the realized policy corresponds to the mode of \(q(\pi)\), but the selection mechanism may be modulated (e.g., via a temperature parameter) to realize sub-optimal or exploratory decision sequences.

Belief: Any internal probability distribution parameterized and maintained by a controller over a random variable (such as representational states \(s\), observations \(o\), or decisions \(\pi\)). It is a strictly operational data structure devoid of conscious semantics. Depending on its role within the control algorithm, a belief functions as:

• Prior belief: A distribution encoding heuristic constraints, habits, or structural goals before evaluating current evidence (e.g., the deontic model \(\tilde p(\pi_{k-1} \mid s_{k-1})\) or the structural setpoint distribution \(\tilde p(s)\)).
• Posterior belief (state estimate): A dynamically updated distribution synthesized from priors and prediction errors (e.g., the recognition distribution \(q(s \mid \theta)\)).
• Predictive belief: A simulated distribution generated via the transition model to evaluate counterfactual trajectories (e.g., the waypoint distribution \(q(s_{[1:K]})\)).

Message: A transient data structure crossing the Markov blanket or moving temporally/laterally between states, containing free energy gradients or distribution parameters \(\theta\) (such as mode and precision).

Instantaneous free energy, variational free energy, free energy, \(\mathcal{F}\), \(\mathcal{F}_{inst}\): The real-time loss function computed by a reflexive controller as an upper bound for instantaneous surprise. It is minimized to drive immediate perception and reflex action. It is evaluated by comparing the current recognition distribution against the normative generative model (the target state):

$$\mathcal{F}_{inst} = D_{KL}[q(s_0 \mid \theta) \parallel \tilde p(s_0)] – \mathbb{E}_{q(s_0 \mid \theta)}[\ln p(o_0 \mid s_0)]$$

Retrospective free energy, \(\mathcal{F}_{retro}\): The sequential loss function computed by a temporal controller over a history of states. It is minimized exclusively through perception (filtering and smoothing) to accurately estimate the past and present based on empirical evidence. It is evaluated by comparing the internal sequence of beliefs against the predictive generative model (the physics engine):

$$\mathcal{F}_{retro} = \mathbb{E}_{q(s_{-N:0}, \pi_{-N:-1})}\left[ \ln q(s_{-N:0}, \pi_{-N:-1}) – \ln p(o_{-N:0}, s_{-N:0}, \pi_{-N:-1}) \right]$$

Expected free energy, \(G(\pi)\): The prospective loss function used during waypoint inference to evaluate a policy \(\pi\) over a future planning horizon. It balances pragmatic value (exploitation) with epistemic value (exploration) by comparing the simulated future generated by the predictive generative model against the goals defined by the normative generative model:

$$G(\pi) = \sum_{k=1}^K \mathbb{E}_{q(o_k, s_k \mid \pi_{0:k-1})} [ \ln q(s_k \mid \pi_{0:k-1}) – \ln \tilde p(s_k, o_k) ]$$

Expected free energy over policies, \(G’\): The prospective loss function defined over the full policy distribution \(q(\pi)\) used during waypoint inference:

Smoothing, postdiction: The process of updating the recognition distribution over past representational states \(q(s_{-N:0})\) and decisions \(\pi_{-N:-1}\) given observations \(o_{-N:0}\) by minimizing retrospective free energy.

Epistemic foraging: Actions taken to minimize state uncertainty (improving the state estimate). Should not be confused with learning or active learning (see below).

Active learning: A realized policy \(\pi^*\) or action \(a\) executed to drive the physical state \(\eta\) into configurations that generate observations \(o\) capable of reducing uncertainty about the generative model parameters \(\phi\). It is driven by the minimization of expected free energy \(G(\pi)\).

Ontogenetic learning: The continuous, internal computational optimization of the generative model parameters \(\phi\) that occurs within an individual organism’s (or controller’s) lifespan. It corresponds to “learning” in standard machine learning and active inference, but is explicitly qualified as ontogenetic to distinguish it from discrete, generational adaptations (phylogenetic structure learning).

It is driven by continuous gradient descent on instantaneous free energy \(\mathcal{F}_{inst}\) or retrospective free energy \(\mathcal{F}_{retro}\), and its velocity is governed strictly by the learning update rate \(K_\phi\). Crucially, ontogenetic learning operates strictly within a fixed generative model structure \(m\); it tunes continuous relationships but cannot alter the dimensionality or topology of the state space.

It serves two simultaneous functions:

• Amortization: It structuralizes the results of recurrent free energy minimization by training the amortized perception model \(\theta(o; \phi)\) and policy-conditioned generative dynamical model \(\dot \mu = f(\mu, \pi; \phi)\) to approximate the outputs of the foundational models. Amortized models shift computational burden from slow, iterative gradient descent to fast, reflexive feedforward sweeps.
• Calibration: It continuously refines the foundational transition model \(p(s_k \mid s_{k-1}, \pi_{k-1})\) and observation model \(p(o \mid s)\) to maintain alignment with the physical environment and the organism’s changing morphology over its lifetime.

Generative model structure, \(m\): The discrete architectural definition of a generative model. It dictates the dimensionality and topology of the state space \(\Omega_s\), the conditional dependencies between state variables (the graph structure), and the mathematical families of the probability distributions. It is the fixed scaffold within which the continuous parameters \(\phi\) operate.

Model evidence, \(p(o \mid m)\): The marginal likelihood of an observation sequence \(o\) given a specific generative model structure \(m\), integrated over all possible states and parameters. It is the metric by which different model structures are evaluated against one another.

Model free energy, \(\mathcal{F}(m)\): The variational free energy evaluated at the level of the model structure, serving as an upper bound on the negative log evidence of the model:

$$\mathcal{F}(m) \approx – \ln p(o \mid m)$$

Where:

\(m\): The discrete generative model structure.

\(o\): The sequence of observations.

\(p(o \mid m)\): The model evidence.

Structure learning (or model selection): The discrete computational process of transitioning from one generative model structure \(m_i\) to another \(m_j\). This occurs when the current model \(m_i\) consistently generates large prediction errors that cannot be resolved via gradient descent on \(\phi\) (meaning the observations persistently fall outside the computational viable manifold \(\mathcal{M}_{s, i}\)). The controller selects a new model structure \(m_j\) from a space of possible structures \(\mathcal{M}_{space}\) such that \(\mathcal{F}(m_j) < \mathcal{F}(m_i)\). Unlike learning, this is not a continuous gradient descent, but a discrete state transition in model space.

Structure generation (or abductive generation): The stochastic or combinatorial process that dynamically creates a novel discrete generative model structure \(m_{new}\) and introduces it into the space of possible structures \(\mathcal{M}_{space}\). It is driven by the destabilizing effects of persistently high free energy when continuous learning on \(\phi\) fails. It involves altering the dimensionality of the state space \(\Omega_s\) or creating novel conditional dependencies between existing high-level representational state variables.

Space of possible structures, \(\mathcal{M}_{space}\): Rather than a static, pre-defined set of all conceivable models, \(\mathcal{M}_{space}\) should be defined as a dynamic, computationally bounded horizon of adjacent possible structures. It is continuously expanded via structure generation and pruned via model selection.

Abductive inference: The complete, discrete macro-process of creativity and paradigm shifting. It encompasses both the rapid structure generation of a novel hypothesis \(m_{new}\) and its immediate model selection based on the minimization of model free energy \(\mathcal{F}(m)\).

Phylogenetic structure learning (natural selection): The ultimate, population-level mechanism of model selection. Over evolutionary time, the physical environment evaluates the model evidence \(p(o \mid m)\) of an entire generation of organisms. If an organism’s endowed generative model structure \(m\) and structural setpoint distribution \(\tilde p_{struct}(s)\) are poorly calibrated to the environment, it fails to maintain NESS and degrades (dies). This failure effectively prunes that specific structure from the space of possible structures \(\mathcal{M}_{space}\) of the species, ensuring that only viable model structures propagate to the next generation.

Phylogenetic structure generation (mutation/recombination): The biological counterpart to abductive generation. The stochastic alteration of an organism’s morphological blueprint (DNA) during reproduction. This injects noise at the phylogenetic level, continuously populating the species’ space of possible structures \(\mathcal{M}_{space}\) with novel generative model structures \(m_{new}\) and shifted structural setpoint distributions \(\tilde p_{struct}(s)\) to be evaluated by the environment.

Ontogenetic learning: A clarifying synonym for your existing definition of learning. It distinguishes the continuous optimization of generative model parameters \(\phi\) (which occurs within an individual organism’s lifespan via the learning update rate \(K_\phi\)) from the discrete structural leaps that occur across generations.

---

The table below summarizes the mapping between the terms above and conventional AIF terms.

Term	AIF equivalent	Role
Controller	Agent; brain	The control system controlling the controlled system by minimizing free energy.
System; controlled system; allostatic system	Generative process	The physical system being controlled.
Physical state	External state	The state of the controlled system.
Controller state; information state	Hidden state; latent variable	The controller’s “guess” at the system state.
Viable manifold	Attracting set	The “safe” operating envelope of physical states.
Setpoint distribution	Prior; prior distribution; prior preferences; \(C\)-matrix	The setpoint or epistemic bias for the controller.
Decision	Action; control variable; \(u\)	Determines the probabilities for the state transitions in transition model calculations.
Waypoint inference	Discrete-time POMDP; sophisticated inference/planning; deep tree search; policy inference	Produces the trajectory of state distributions for reaching the setpoint distribution.
Deontic model	Prior over policies; habits; \(E\)-matrix	Prior preferences over policies such as habits or moral rules.
Normative generative model	Target joint distribution / Generative model (when applied to priors)	Defines target regions and expected sensory observations to act as the mathematical baseline for free energy minimization.
Predictive generative model	Empirical generative model / Tree-search model / Instantiated generative model	Simulates prospective state sequences over a planning horizon and infers past states based on transition physics.

Artificial intelligence

Artificial intelligence (AI): The capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. “AI” is often used informally as a noun denoting software components with artificial intelligence.

Machine learning: A field of study in artificial intelligence concerned with the development and study of statistical algorithms that can learn from data and generalise to unseen data and thus perform tasks without explicit instructions.

Algorithm: A computational architecture or procedural method that can be trained on data to perform a task. Examples: convolutional neural network, transformer.

Training algorithm: A set of instructions used in machine learning to iteratively adjust the parameters of a model so it can learn from data and make accurate predictions on new data.

Model: A specific, trained AI algorithm. Example: GPT-4o.

Inference engine: A software and/or hardware component that evaluates a trained model on new data and produces output results (e.g., predictions). Example: TensorFlow.

AI application: A system in which one or more critical or high-value functions are realized by an AI component. Examples: ChatGPT, GitHub Copilot, autonomous vehicle

Systems engineering

System: An arrangement of interacting elements that together realize functions. Examples include information systems, biological organisms, ant hills, medical devices, medical laboratories, and space stations.

A system can be decomposed into (sub)systems that in turn can be decomposed into even lower-level systems, components, and ultimately parts. The term can in a specific context refer to systems as-designed (system “blueprints”), systems as-built (constructed, manufactured system), and as-maintained (operational, real-world systems).

System element: System, subsystem, component, or part.

Subsystem: A system that is part of a larger system. This term is only used informally.

Component: A system element that encapsulates a cohesive responsibility, realized by a minimal sufficient set of functions, and that interacts with its environment via a well-specified contract (its provided and required interfaces). If the contract is preserved, the component is substitutable.

Part: The lowest level of system elements for which configuration information is defined.

Risk management

Harm: Physical injury or damage to the health of people, or damage to property or the
environment.

Hazard: Potential source of harm.

Hazardous situation: Circumstance in which people, property, or the environment are
exposed to one or more hazard(s).

Risk: Combination (usually multiplication) of the probability of occurrence of harm and the
severity of that harm. Reduction of risk is the objective of risk management. A risk can be
reduced either by lowering the severity of the harm or the probability of the harm, or both.
The severity depends on the nature of the hazardous situation.

Risk control: Process in which decisions are made and measures implemented by which
risks are reduced to or maintained within specified levels. Risk control measures are also
called risk mitigations.

Fault: Condition or defect in a system which may lead to an error. Synonym: defect, bug.

Error: Manifestation of a fault as an unexpected or unwanted behavior of a system. An error
may lead to a failure.

Failure: Situation in which a system (or part of a system) is not performing its intended
function due to an error. The failure is characterized by its failure mode i.e. the specific
manner in which the failure occurs. A failure is a symptom of an error that may or may or
may not lead to a hazard.

Failure mode: The specific manner or way in which a failure occurs.

Configuration management

Configuration item (CI): A set of functional, performance, and physical characteristics of a system element; a set of true statements about a system element that we choose to manage as a unit. This is the most important term in configuration management.

A configuration item has an extension in time. It comes into existence at one point in time, e.g., when it is defined in the systems engineering process, and it goes out of existence, e.g., when the whole system reaches end of life or when the configuration item is replaced by a new configuration item because of a new system architecture.

Configuration items usually evolve over time as the system is improved (except as-built configuration items, see below). The evolution of the configuration item is documented as a sequence of configuration item versions, each of which defines the configuration information during a defined period of time, the validity period.

Configuration item versions are in turn documented in the *configuration information*, a set of documents, models etc, associated with the configuration item (see Figure 1a and Figure 3).

In the most general case, configuration management spans over the following types of configuration items:

As-designed configurations item: (Predicted) characteristics of designed (but not necessarily yet constructed) system element.

As-built configurations items: (Actual and predicted) characteristics of constructed (but not yet put in operation) system element.

As-maintained configuration item: (Actual) characteristics of a system element in operations.

A configuration item should be selected so that it is realized by one system or system element. Configuration items should furthermore be selected so that they can be managed with minimal dependency on other configuration items (modularity). Configuration item selection criteria should therefore consider:

• Regulatory requirements
• Criticality in terms of risks and safety
• Anticipation of new technology replacing old
• Interfaces with other configuration items
• Procurement
• Support and service

Examples:

• Components that are procured as a single entity (e.g., an x-ray tube) should be represented by one configuration item that defines the specifications of the component.
• System elements that have their own maintenance schedule and maintenance log should be represented by one configuration item that is all the maintenance-related information.
• System elements that are developed as one entity, with their own specifications, risk analyses etc. should map to one configuration item.

Configuration: A top-level (root) configuration item; the characteristics of a system element that is managed as an independent entity, often a whole system.

Configuration item version (CIV): A (long exposure time) snapshot of a configuration item between two timestamps, `valid_from` and `valid_to`.

If `valid_to` is not set or set to a future date, then the configuration item is currently valid.

For as-designed configuration items valid means released, i.e., formally approved for construction after `valid_from`.

If the last version of an as-designed configuration item has its `valid_to` in the past then it is deleted from the configuration, i.e., the corresponding system element is no longer part of the system.

As built configuration items are kept valid for as long as the system is assumed to be on the market or, in case of medical devices, sometimes indefinitely.

For as-maintained configuration items valid means installed, i.e., operational in a system instance.

An as-maintained configuration item with its `valid_to` in the past is removed from the operational system.

Key configuration item versions (baselines) can also be given a human-readable label, like v2.1.0.

Configuration version: A snapshot of a configuration at a well-defined point in time; a top-level configuration item version.

Configuration baseline: A formally reviewed and approved configuration version that serves as a non-volatile reference point for a well-defined purpose.

Configuration item graph: A directed acyclic graph (DAG) of configuration items. The graph mirrors the structural decomposition of the system of interest. The reason that the CI graph is not a simple tree is that the same configuration item (e.g., a power outlet) may appear in many branches of the (as-built) configuration item graph in an as-designed configuration. The “acyclic” adjective means that a configuration item can not be part of itself; there can not be any circular part-of associations.

An example of a CI graph is shown in the figure below. Note that Part 3 is part of both Component 1 and Component 2.

Configuration information: A representation of a configuration or a configuration item. Typically a set of documents, source code, models, database records etc that together document the characteristics of a system element (a configuration item).

Information item: A repository of configuration information such as a document, a model, a source code file, or a database record. Information items evolve over time. Their state at a certain point in time is represented by an information item version, a snapshot of the information item at that point in time.

Information item version (IIV): A snapshot of an information item at a well-defined point in time.

Change request: A formal request to change a baseline, e.g., by adding a new function to the system. The change request is not part of the configuration information since it describes a difference between two configuration baselines rather than a configuration baseline.