Terminology

Nomina si nescis, perit et cognitio rerum.
(If you know not the names of things, the knowledge of things themselves perishes.)

Carl von Linné

Active inference framework

General notes and guidelines

• Time sequences: Time sequences are indexed with the subscript \(k\) where the subscript \(0\) represents the present time. Negative numbers represent the past and positive numbers represent the future. The time intervals between the time samples are not assumed to be equidistant.
• Shorthand: A random variable without a subscript, e.g., \(s\), is a shorthand for either a a random variable at a certain point in time or a time sequence of the random variable, as given by the context. When explicitly referring to a random variable at a specific time sample, a single subscript is used, e.g., \(s_0\). When a defined range need to be defined, a notation like \(s_{-N:0}\) is used.
• Tilde: A tilde over a probability distribution, like \(\tilde p(s)\), \(\tilde p(o)\), or \(\tilde p(\pi)\) means that the distribution is a normative distribution, rather than a neutral prediction. It defines what ought to be in rather than what is.
• Controller focus: Active inference is more focused on the controller than the controlled system. Therefore terms without a “controlled system” qualifier mean quantities associated with the controller. Thus when we refer to a “state” without any qualifier, it means controller (representational) state. When referring to the controlled system state we write “controlled-system state” or make the meaning otherwise clear from the context.
• Term order: While the goal has been to order the definitions from atomary and primitive terms to composite and dependent terms, this has not always succeeded. Some scrolling both up and down may be needed to fully understand a term.

System: An arrangement of interacting elements that together realize functions. Examples include information systems, biological organisms, ant hills, medical devices, and space stations.

Open system: A system that has external interactions. Such interactions can take the form of information, energy, or material transfers into or out of the system boundary.

Distribution, probability distribution: A mathematical function or table that describes the likelihood of all possible outcomes for a random variable, discrete or continuous. Quantified with a probability density function (continuous random variables) or a probability mass function (discrete random variables).

State vector: An ordered array of state variables that mathematically defines a single point within a state space \(\Omega\). Within active inference, a state vector represents either a controlled-system state or an information state within a controller, a representational state.

State variable: A component of a state vector. State variables can be categorical or continuous. Controllers in contact with or close to the biological system work with continuous state variables such as joint angle and body temperature. Higher-level (human) controllers work with more abstract and often categorical state variables such as \(\texttt{mug of coffee in hand}\) and \(\texttt{social status}\).

State: The current value of the state vector.

State space, \(\Omega\): The bounded, multi-dimensional mathematical space comprising all permissible state vectors for a given controller or controlled system. Its dimensions are spanned by the domains of its constituent state variables. Any specific state vector, e.g., a representational state \(s\) or a controlled system state \(\eta\), is a single point within this coordinate system, uniquely defined by the simultaneous values of its constituent state variables.

Controlled system: The system on the external side of a chosen controller’s interface, whose states the controller influences through effectors and observes through transducers. It may itself comprise nested controllers and controlled systems, ultimately implemented by molecular dynamics that need not themselves constitute further controllers.

Examples of controlled systems include individual cells, tissue and organs, the whole human body, and the habitat of organisms capable of modifying their habitat.

Controller, \(\mathcal {C}^{(i)}\): The system on the internal side of a chosen interface that uses observations to infer the controlled system’s state and influences it through actions to bring observations into conformity with its setpoint distribution \(\tilde p(s)\) or \(\tilde p(o)\). Its setpoint distributions may be a combination of intrinsic setpoint distributions and setpoint distributions supplied by other controllers. It may itself be part of a controlled system relative to another controller. The superscript \(i\) indicates the level of the controller in a hiearchy (“stack”) of controllers.

Controlled-system state, \(\eta\): Vector of state variables that describe the controlled system’s state. When the controlled system is a physical body, the controlled-system state describe mechanical, chemical, thermodynamic, or other directly physically instantiated quantities. When the controlled system is another controller the controlled-system state is the representational state of that controller. This means that the representational state of \(\mathcal {C}^{(i)}\) is the controlled-system state from the perspective of \(\mathcal {C}^{(i+1)}\).

Macrostate, \(z\): A state variable that summarizes, coarse-grains, or otherwise represents a configuration of several individual state variables. An example of a macrostate is \(\texttt{social status}\).

$$z = g(\eta)$$

Generative process: The dynamics of a controlled system. It is the process that the controller’s predictive generative model attempts to simulate. It is defined by the continuous time evolution of the controlled system state \(\eta\):

$$d\eta = f(\eta, \eta_a)dt + d\omega_\eta$$

where

• \(\eta_a\) is an action state generated through an effector by the controller controlling the generative process.
• \(\omega_{\eta}\) is a random disturbance of the controlled system state.

The evolution of macrostates \(z\) can often be described as discrete time process:

$$p(z_k \mid z_{k-1}, u_{k-1})$$

where \(u\) is a macroaction consisting of a trajectory of action states.

Control: In active inference, acting to bring observations into conformity with a setpoint distribution (preferred distribution) defined in terms of observations \(\tilde p(o)\) or in terms of representational states \(\tilde p(s)\). When adaptive, such control contributes to maintaining the controlled system state within its viable region—the region of states that supports its continued self-maintenance far from thermodynamic equilibrium.

Predictive control: Control informed by a model of the controlled system’s generative process enabling the controller to predict future controlled system states over a time horizon and choose the most favorable sequence of actions (policy) going forward.

Active inference: A general form of predictive control encompassing perception, action, planning, decision-making and learning, employed by biological organisms.

Viable region, \(\mathcal{M}_v\): The set of states in which a controlled system maintains its function and structural integrity. Depending on the level of description, these include physical conditions—such as adequate energy supply, structural integrity and workable chemical concentrations—and computational conditions—such as maintaining coherent, sufficiently accurate representations and the capacity to process information, infer and regulate action.

Organism: A biological system.

Closed-loop controller: A controller that utilizes a continuous stream of observations \(o\) to update its representational distribution \(q(s)\) and to generate actions \(a\) to keep a controlled-system state near its setpoint state.

Open-loop controller: A controller that executes actions \(a\) without updating its representational distribution \(q(s)\) from real-time observations \(o\) during the execution. The computational burden has been amortized into a policy-conditioned generative dynamical model through learning:

$$\dot s = f(s, \pi)$$

where \(\pi\) is the intended policy, e.g., moving the location of a golf ball from the tee to the intended spot on the fairway.

In active inference, very fast actions must be open-loop because of observation-error calculation-action latency.

Controller network, \(\mathcal{C}\): A network of interacting controllers separated by interfaces, collectively controlling an organism. An organism may have several controller networks for different purposes. The controller networks have a layered architecture.

Source controller, \(\mathcal{C}_s\): A controller partially controlling a target controller by setting the empirical setpoint distribution of the target controller.

Target controller, \(\mathcal{C}_t\): A controller receiving empirical setpoint distributions from one or more source controllers. A target controller combines all received empirical setpoint distributions with its own structural setpoint distribution to a total (combined) setpoint distribution that it then pursues:

$$\tilde p_{total}(s) \propto \tilde p_{struct}(s) \prod_{i=1}^I \tilde p_{emp, i}(s)$$

Terminal controller: A target controller at the boundary of the controller network \(\mathcal{C}\) that has no subordinate target controllers. It constitutes the barrier where the informational domain, and therefore active inference, ends and the unguided physical domain (thermodynamics and mechanics) begins.

Regulator: A controller that operates strictly in the current moment. It receives a real-time stream of observations. From the observation in any given moment it infers a representational distribution and an action to minimize (variational) free energy using its generative normative model, also in (near) real-time. A regulator doesn’t have any memory and it doesn’t plan ahead. While it does not have memory of past observations, it can internally represent the history of the state via the generalized coordinates of motion estimated in the current moment.

Planner: A controller that operates over the time sequence \(-N:K\) where 0 represents the current moment.

• It infers a representational distribution \(q(s_{-N:0})\) from past and current observations \(o_{-N:0}\) minimizing retrospective free energy over its retrospection horizon. Weighing in past observations renders a more accurate current representational distribution than can be inferred by a regulator only considering the current observation.
• It infers a target distribution \(q(s_{1:K} \mid \pi^*)\) to guide future actions by evaluating one or more alternative policies \(\pi\) from its policy space \(\mathcal{U}(s)\) and selecting a policy \(\pi^*\) that minimizes a loss function called expected free energy (EFE) over its planning horizon.

Controlled-system setpoint state, \(\tilde \eta\): The instantaneous ideal state of a controlled system at any given moment; the mode of the controlled-system setpoint distribution \(\tilde p(\eta)\). It corresponds precisely to the physical setpoint or reference state in classical control theory.

Controlled-system setpoint distribution, \(\tilde p(\eta)\): The probability distribution over the controlled-system state \( \eta\) that the controlled system is currently biased to occupy. It is the controlled-system counterpart of the controller’s internal setpoint distribution \(\tilde p(s)\).

Information state, \(s, o, a\): A state vector whose state variables are random variables representing quantities of interest within a controller. The three types of information states in active inference are:

• Observation, \(o\)
• Action, \(a\)
• Representational state, \(s\)

Out of these, the action and the observation are point estimates. The controller infers an implicit precision of the observation which is used the observation model to modulate perceptual updating (an observation in a snowstorm has low precision). Because observations aren’t perfect, there will always be uncertainty in the estimation of the current representational state. The representational state is therefore represented as a probability distribution \(q(s)\) rather than a point estimate.

Representational state, \(s\): The random variable that spans the controller’s internal state space \(\Omega_s\). It serves as the mathematical proxy for the controlled-system state \(\eta\). It acts as the continuous or categorical domain (support) over which the controller evaluates the representational distribution \(q_{\theta}(s)\) and the setpoint distribution \(\tilde p(s)\).

Representational state variables vary in their level of abstraction:

Low-level representational state variables represent immediate, localized physical states of the controlled system alongside their temporal derivatives up to a given order (typically order 2, capturing velocity and acceleration).

High-level representational state variables represent compressed, complex, long-term physical states and controlled-system macrostates such as \(\texttt{mug of coffee in hand}\), \(\texttt{social status}\), and \(\texttt{destination of flight}\).

The representational state is often called “hidden state” or “latent state” in canonical active inference literature. While “latent state” is mathematically correct since the value of the state is generally not accessible as a point estimate, we prefer the functional term representational state.

Representational distribution, \(q_{\theta}(s)\), \(q_{\theta}(s, \pi)\):

\(q_{\theta}(s)\) is a variational distribution that acts as a tractable approximation of the intractable posterior distribution \(p(s \mid o)\) in a regulator. It is the best estimate of the current representational state of the controller.

\(q_{\theta}(s, \pi)\) is a variational distribution that acts as a tractable approximation of the intractable posterior distribution \(p(s, \pi \mid o) = p(s \mid \pi, o)p(\pi \mid o)\) in a planner. \(s\) and \(\pi\) are past sequences of the respective random variables.

\(\theta\) represents the recognition distribution parameters:

• If \(q_{\theta}(s) = \mathcal{N}(s; \mu, \Pi^{-1})\) (the Laplace approximation), then \(\theta = (\mu, \Pi)\).
• If \(q_{\theta}(s) = \text{Categorical}(\theta)\), then \(\theta\) is an array of probabilities, e.g., \(\theta=[P(\text{tea}), P(\text{coffee}), P(\text{water})]\).

\(\theta\) is the variable available for the control algorithm in the controller. In summary:

$$ \underbrace{s}_{\text{representational state}} \qquad
\underbrace{q_{\theta}(s)}_{\text{representational distribution}} \qquad
\underbrace{\theta}_{\text{algorithmically available representation}}$$

Generalized coordinates of motion: Representational state vector representing a dynamic process, e.g., motion or shifting concentration. Its state variables represent the quantity, the time derivative of the quantity, the second time derivative of the quantity etc. For motion in the x-direction the generalized coordinates of motion would for instance be:

$$s = \begin{bmatrix} x \\ \dot x \\ \ddot x \\ \vdots \end{bmatrix}$$

Where \(x\) is the current position of the object in x-direction, \(\dot x = dx / dt\) etc.

Surprise, \(– \ln \tilde p(o)\): The negative logarithm of the probability of an observation as given by a normative generative model. Active inference controllers are biased to minimize surprise.

Observable state, \(\eta_o, s_o\): A vector of values \(\eta_o=h(\eta)\) available as inputs to a specified sensory transducer. \(h(\eta)\) selects or combines controlled-system state variables. Transduction produces an observation \(o\), represented deterministically as \(o=T(\eta_o)\).

Within a controller network, the observable state of a target controller \(s_{o, t}\) is the full set of parameters \(\theta_t\) defining its representational distribution \(q_{{\theta}_t}(s_t)\).

Observation, \(o\): A sequence of point-estimate information states. At the physical boundary, it is derived via transducers from the observable state \(\eta_o\).

Within a controller network, the observation \(o_s\) received by a source controller equals the representational state of the target controller’s observable state \(s_{o, t}\).

Secondary parameters from \(s_{o, t}\), such as precision \(\Pi_t\), bypass the observation vector to directly parameterize the variance of the source controller’s observation model \(p(o_s \mid s_s; \Pi_t)\).

Action state, \(\eta_a, s_a\): In a physical system, a continuous physical state \(\eta_a\) generated by an effector with the intention to force or maintain the system’s physical state, counteracting environmental dynamics. It is the physical realization of the controller’s action \(a\), transformed through an effector to a continuous force or flux:

$$d\eta = f(\eta, \eta_a)dt + d\omega_\eta$$

Within a controller network, the action state of a target controller \(\mathcal{C}_t\) from the perspective of a source controller \(\mathcal{C}_s\) is the setpoint distribution \(\tilde p_t(s_t)\) generated by the \(\mathcal{C}_s\) action \(a_s\).

Macroaction, \(u\): A coarse-grained, usually temporally extended change or activity of the controlled system that is realized through a trajectory of physical action states that influence one or more macrostates.

Action, \(a\): A continuous information state that drives an action state in the controlled system via an effector. The action state is a function of the action:

$$\eta_a = h(a)$$

Markov blanket \(b\): The set of boundary states \(\eta_b = \{\eta_o, \eta_a\}\). The Markov blanket mediates all information between the controller states and controlled system states.

Mathematically the Markov property implies that

$$p(s, \eta | \eta_b) = p(s | \eta_b)p(\eta | \eta_b)$$

Markov blanket is a key concept in active inference theory but its role is contested and it is not necessary to explain active inference as a control theory.

Agency model, \(\frac{\partial o}{\partial a}\): The controller’s internal mapping that defines its capacity to alter the controlled system state and therefore its observations \(o\) by emitting an action \(a\) to drive effectors across the Markov blanket. It serves as the mathematical gatekeeper determining whether a free energy gradient is resolved via action (when agency \(\neq 0\)) or forced into perception/acceptance (when agency \(= 0\)).

Observation model, \(p(o \mid s)\): The controller’s mapping between representational states \(s\) and expected observations \(o\). It defines how the controller expects a given physical state represented by \(s\) to manifest as sensory data (transformed by transducers).

State transition model, \(p(s_k \mid s_{k-1}, \pi_{k-1})\): The “physics engine” of a planner modeling the controlled system dynamics. It dictates how the controller expects the representational state \(s_{k-1}\) to evolve into the next state \(s_k\) as a consequence of a virtual action \(\pi_{k-1}\) which is one step of the policy \(\pi\).

Deontic model, \(\tilde p(\pi)\): Dictates the controller’s prior beliefs regarding which policies are afforded (believed to be physically executable), historically habitual, or otherwise normative.

Amortized inference: inference in which a learned feed-forward mapping \(\theta(o; \phi)\) takes observations directly to an approximate posterior (or its sufficient statistics), avoiding a fresh optimization for each observation. It is fast but approximate, because the inference computation has effectively been learned in advance.

Iterative inference: inference in which the current posterior parameters, e.g., the parameters of the representational state \(q_{\theta}(s)\) are repeatedly updated to reduce variational free energy for the present observation, typically by gradient descent on free energy. It is slower than amortized inference but adaptive, because inference is solved anew—or refined—online.

Update rate, \(K_\theta, K_a, K_\phi\): The rate constant matrices (with dimension \([\text{time}]^{-1}\)) that govern the velocity of free energy gradient descent. They act as the physiological gain translating computational prediction errors into parameter updates or physical forces or fluxes.

• Perception update rate, \(K_\theta\): Dictates the rate of change of the representational distribution parameters \(\theta\).
• Action update rate, \(K_a\): Dictates the rate of change of the action state \(a\).
• Learning update rate, \(K_\phi\): Dictates the rate of change of the generative model parameters. Typically much smaller than \(K_\theta\) and \(K_a\).

Amortized perception model, \(\theta_{\phi}(o)\): A fast learned inverse model that instantly outputs the approximate parameters of the representational distribution \(q_{\theta}(s)\) based on current sensory input. This model allows a controller to perform a rapid feedforward sweep to immediately minimize instantaneous free energy \(\mathcal{F}_{inst}\), bypassing the computational latency of iterative inference. Amortized perception may be combined with and fine-tuned with iterative inference.

Amortized action model, \(\dot s = f_{\phi}(s, \pi)\): A learned mapping parameterized by \(\phi\) that guides a ballistic or semi-ballistic movement such as a golf swing. This model allows a controller to execute goal-directed physical changes when the real-time observation model \(p(o \mid s)\) is partially or fully inactive,e.g., due to timing constraints, bypassing or accelerating iterative inference.

Setpoint state, \(\tilde s\): A point in the controller state space (\(\tilde s \in \Omega_s\)) that the controller is configured to achieve and maintain. It serves as the reference value toward which the controller’s physical actions or internal perceptual biases are directed. It corresponds to the mode of the setpoint distribution:

$$\tilde s = \underset{s}{\arg\max} \, \tilde p(s)$$

Note that for Gaussian distributions the mode equals the mean. Categorical distributions don’t have a well-defined mean but they have a mode (a most probable category).

Setpoint distribution, \(\tilde p(s), \tilde p(o)\): A probability distribution over representational states or observations that defines the target regions the controller is biased to occupy. Mathematically, it occupies the position of a Bayesian prior. Functionally, it fulfills two simultaneous roles depending on the gating of the update rates:

• Pragmatic target (action): The setpoint distribution drives action to shift the controlled system’s state toward the the physical state represented by the setpoint state \(\tilde{s}\) (the mode of \(\tilde p(s)\)).
• Epistemic bias (perception): The setpoint distribution acts as a continuous perceptual expectation, pulling the representational distribution \(q_{\theta}(s)\) toward it.

The total setpoint distribution steering the controller is the integrated product of two components:

1. Structural setpoint distribution, \(\tilde p_{struct}(s)\): Hardcoded, phylogenetically or architecturally endowed probability distribution with high probabilities allocated to preferred representational states on the viable manifold (e.g., a distribution with a sharp peak at the optimal human body temperature of 37°C).
2. Empirical setpoint distribution, \(\tilde p_{emp}(s)\): Probability distribution dynamically generated by a higher-level source controller and passed down to serve as a pragmatic target or epistemic bias in the target controller. When passed from a planner calculating in discrete time steps to a continuously operating regulator, the parameters of this distribution (such as the setpoint state \(\tilde s\)) are held constant as continuous reference values between discrete update intervals.

Setpoint precision, \(\tilde \Pi\): The precision of the setpoint distribution.

Setpoint observation, \(\tilde o\): The mode of the setpoint distribution when the support is observations. This is is not often used in active inference since the setpoint distribution over observations is genereally intractable.

Prediction error: The mathematical discrepancy between the empirical observation \(o\) and the expected observation, given the current recognition distribution \(q_{\theta}(s)\) and the observation model \(p(o \mid s)\). In a regulator, it corresponds to the gradient of the instantaneous free energy \(\mathcal{F}_{inst}\). It serves as the direct computational signal that, when multiplied by the respective constant update rates (\(K_\theta\) and \(K_a\)), dictates the velocity of updating of \(q_{\theta}(s)\) and physical action.

Generative model, \(\tilde p(s, o), p(s, o, \pi)\): Mathematical constructs and shorthands used to describe active inference algorithms. Active inference defines two types of generative models:

• Normative generative models
• Predictive generative models

Normative generative model, \(\tilde p(s, o)\): A mathematical shorthand representing the joint probability distribution over states and observations that the controller is biased to occupy. It does not exist as a single biological or physical structure; rather, it is the computational product of evaluating the controller’s observation model \(p(o \mid s)\) and its setpoint distribution \(\tilde p(s)\) simultaneously. It serves as the theoretical target for free energy minimization:

$$\tilde p(s, o) = p(o \mid s) \tilde p(s)$$

Planning horizon: The future horizon over which a planner plans infers a policy. It extends over the virtual actions \(\pi_0\) to \(\pi_{K-1}\) and thus the representational states \(s_1\) to \(s_K\). The planning horizon is defined by the upper index \(K\).

Retrospective horizon: The past horizon over which a planner infers its recognition distribution \(q(s_{-N:0}, \pi_{-N:-1})\). The retrospective horizon is defined by the lower index \(N\).

Predictive generative model, \(p(s, o, \pi)\): A model of the generatieve process of the controlled system. It is used both for analyzing the past to establish the present representational state and simulating the future to infer the best course of actions as will be shown below. In a biological controller it is factorized into a deontic model, an observation model, and a transition model:

$$p(s, o, \pi) = \tilde p(\pi) p(s, o \mid \pi) = \tilde p(\pi) p(o \mid s, \pi) p(s \mid \pi)$$

$$p(s_{0:K}, o_{0:K}, \pi_{0:K-1}) = \tilde p(\pi) p(s_0) \prod_{k=0}^K p(o_k \mid s_k) \prod_{k=1}^K p(s_k \mid s_{k-1}, \pi_{k-1}) $$

$$p(s_k, o_k, \pi_{k-1} \mid s_{k-1}) = \tilde p(\pi)p(o_k \mid s_k) p(s_k \mid s_{k-1}, \pi_{k-1})$$

$$p(s_{k} \mid \pi_{k-1}) = \sum_{s_{k-1}} p(s_{k} \mid s_{k-1}, \pi_{k-1}) p(s_{k-1})$$

$$p(s \mid \pi) = p(s_0) \prod_{k=1}^{K} p(s_{k} \mid \pi_{k-1})$$

Notation used above:

• \(s_{0:K}\): Sequence of representational states.
• \(o_{0:K}\): Sequence of observations.
• \(\pi_{0:K-1}\): Sequence of virtual actions; policy.
• \(p(s_0)\): Current representational state distribution, approximated by \(q_{\theta}(s_0)\).
• \(\tilde p(\pi)\): Deontic model defining prior preferences over policies.
• \(p(o \mid s)\): Observation model predicting observations conditional on representational states.
• \(p(s_k \mid s_{k-1}, \pi_{k-1})\): Transition model predicting state evolution based on the previous state and state transition.

The predictive generative model for perceptual inference over the retrospective horizon is:

$$p(s_{-N:0}, o_{-N:0}, \pi_{-N:-1}) = \tilde p(\pi) p(s_{-N}) \prod_{k=-N}^0 p(o_k \mid s_k) \prod_{k=-N}^{-1} p(s_{k+1} \mid s_{k}, \pi_{k})$$

with

$$p(s_{-N}) = \mathbb{E}_{q(s_{-N-1}, \pi_{-N-1})}[p(s_{-N} \mid s_{-N-1}, \pi_{-N-1})]$$

Target state, \(s\): A sequence of future states the planner is biased to visit over the planning horizon. The target state is a decomposition of the setpoint state into intermediate states to visit. While there is only one setpoint state in a planning horizon, there can be a whole sequence of target states. The target state is the mode of a target state distribution. The target state is a function of the optimal policy \(\pi^*\) selected in policy inference.

Target distribution, \(q(s \mid \pi^*)\): A distribution of a target states conditional on the optimal policy \(\pi^*\) selected in policy inference.

Policy inference: The process of deriving an optimal policy and a target state distribution. Referred to as planning in control theory.

Generative model parameters, \(\phi\): The parameters (analogous to weights in deep learning) defining the generative model. Learning is defined as the optimization and updating of \(\phi\).

Virtual action, \(\pi_k\): An action-like variable used by a generative model to propagate a predicted state distribution to a subsequent state distribution; unlike an effector action, it is not itself a physical control signal and it doesn’t need to be on the same level of abstraction as an action \(a\). The abstraction level is a modeling choice.

It acts as a parameter altering state transition probabilities:

$$p(s_k \mid s_{k-1}, \pi_{k-1}) = B_{\pi_{k-1}} \Rightarrow q(s_k \mid \pi_{k-1}) = \sum_{s_{k-1}} p(s_k \mid s_{k-1}, \pi_{k-1}) q(s_{k-1})$$

Where:

• \(B_{\pi_{k-1}}\): Transition matrix representing the state transition probabilities from time sample \(k-1\) to \(k\) under decision \(\pi_{k-1}\).
• \(q(s_{k-1})\): Internal distribution over states at the previous time sample.

Note on standard literature: Virtual action is frequently denoted as action in traditional active inference literature, overloading the term with the term for the actual action generated by a controller \(a\).

Policy, \(\pi\): A sequence of virtual actions \(\pi = (\pi_0, \pi_1, \dots, \pi_{K-1})\) drawn from a bounded policy space (\(\mathcal U\)). An optimal policy \(\pi^*\) is selected for execution by the controller based on the expected free energy associated with each policy.

Policy space, \(\mathcal U\): The bounded set of all possible policies available to a controller in a specific context. Baseline active inference doesn’t explain how the policy space is defined. One alternative is to make the policy space conditional on the current state and the setpoint state: \(\mathcal U = h(s, \tilde s)\).

Optimal policy, \(\pi^*\): A policy drawn from \(q(\pi)\) and selected for execution by the controller. The baseline rule is to select the mode of \(q(\pi)\), the most likely policy, for execution as this policy is associated with the lowest expected free energy, but the selection mechanism may be modulated (e.g., via a temperature parameter) to realize sub-optimal or exploratory policies.

Belief: Any controller-internal probability distribution parameterized and maintained over a random variable (such as representational states \(s\), observations \(o\), or policies \(\pi\)). Depending on its role within the control algorithm, a belief functions as:

• Prior belief: A distribution encoding heuristic constraints, habits, or structural goals before evaluating current evidence (e.g., the deontic model \(\tilde p(\pi)\) or the setpoint distribution \(\tilde p(s)\)).
• Posterior belief (state estimate): A dynamically updated distribution synthesized from priors and prediction errors (e.g., the recognition distribution \(q_{\theta}(s)\)).
• Predictive belief: A simulated distribution generated via the transition model to evaluate counterfactual trajectories (e.g., the target distribution \(q(s_{1:K})\)).

We avoid using this term as it comes with a whiff of dualism: if there is a belief, there must be a believer.

Message: A transient data structure crossing the Markov blanket or moving temporally/laterally between states, containing free energy gradients or distribution parameters \(\theta\) (such as mode and precision).

Free energy, instantaneous free energy, variational free energy, \(\mathcal{F}\), \(\mathcal{F}_{inst}\), VFE: The real-time loss function computed by a regulator as an upper bound for instantaneous surprise. It is minimized to drive immediate perception and reflex action. A common expression is:

$$\mathcal{F}[q_{\theta}; o] = D_{KL}[q_{\theta}(s) \parallel \tilde p(s)] – \mathbb{E}_{q_{\theta}(s)}[\ln p(o \mid s)]$$

where \(s\) and \(o\) are the instantaneous values of respective variable.

Retrospective free energy, \(\mathcal{F}_{retro}\): The sequential loss function computed by a planner over a history of states. It is minimized exclusively through perception (as the past cannot be changed by action) to accurately estimate the past and present based on empirical evidence. It is evaluated by comparing the internal sequence of distributions against the predictive generative model (the physics engine):

$$\mathcal{F}_{retro} = \mathbb{E}_{q(s_{-N:0}, \pi_{-N:-1})}\left[ \ln q(s_{-N:0}, \pi_{-N:-1}) – \ln p(o_{-N:0}, s_{-N:0}, \pi_{-N:-1}) \right]$$

Expected free energy, \(G(\pi)\): The loss function used in policy inference to evaluate a policy \(\pi\) over a future planning horizon. It balances pragmatic value (exploitation) with information gain (exploration) by comparing the simulated future generated by the predictive generative model against the goals defined by the normative generative model:

$$G(\pi) = \sum_{k=1}^K \mathbb{E}_{q(o_k, s_k \mid \pi_{0:k-1})} [ \ln q(s_k \mid \pi_{0:k-1}) – \ln \tilde p(s_k, o_k) ]$$

Expected free energy over policies, \(G’\): The loss function for inferring the full policy distribution \(q(\pi)\) in policy inference:

Smoothing, postdiction: The process of updating the recognition distribution over past representational states \(q(s_{-N:0})\) and virtual actions \(\pi_{-N:-1}\) given observations \(o_{-N:0}\), by minimizing retrospective free energy.

Epistemic foraging: Actions taken to gain information to minimize state uncertainty (improving the state estimate). Should not be confused with learning or active learning (see below).

Ontogenetic learning: The continuous optimization of the generative model parameters \(\phi\) that occurs within an individual organism’s (or controller’s) lifespan. It corresponds to “learning” in standard machine learning but is explicitly qualified as ontogenetic to distinguish it from discrete, generational adaptations (phylogenetic structure learning).

It is driven by continuous gradient descent on instantaneous free energy \(\mathcal{F}_{inst}\) or retrospective free energy \(\mathcal{F}_{retro}\), and its velocity is governed by the learning update rate \(K_\phi\). Ontogenetic learning operates within a fixed generative model structure \(m\); it cannot alter the dimensionality or topology of the state space.

It serves two simultaneous functions:

• Amortization: It structuralizes the results of recurrent free energy minimization by training the amortized perception model \(\theta_{\phi}(o)\) and policy-conditioned generative dynamical model \(\dot s = f_{\phi}(s, \pi)\). Amortized models shift computational burden from slow, iterative inference to fast amortized (feed-forward) inference.
• Calibration: It continuously refines the state transition model \(p(s_k \mid s_{k-1}, \pi_{k-1})\) and the observation model \(p(o \mid s)\) to maintain alignment with the physical environment and the organism’s changing morphology over its lifetime.

Active learning: A selected policy \(\pi^*\) executed to drive the physical state \(\eta\) into configurations that generate observations \(o\) capable of reducing uncertainty about the generative model parameters \(\phi\). Active learning is driven by the minimization of expected free energy \(G(\pi)\).

Generative model structure, \(m\): The discrete architectural definition of a generative model. It dictates the dimensionality and topology of the state space \(\Omega_s\), the conditional dependencies between state variables (the graph structure), and the mathematical families of the probability distributions. It is the fixed scaffold within which the continuous parameters \(\phi\) operate.

Model evidence, \(p(o \mid m)\): The marginal likelihood of an observation sequence \(o\) given a specific generative model structure \(m\), integrated over all possible states and parameters. It is the metric by which different model structures are evaluated against one another.

Model free energy, \(\mathcal{F}(m)\): The variational free energy evaluated at the level of the model structure, serving as an upper bound on the negative log evidence of the model:

$$\mathcal{F}(m) \approx – \ln p(o \mid m)$$

Where:

• \(m\): The discrete generative model structure.
• \(o\): The sequence of observations.
• \(p(o \mid m)\): The model evidence.

Structure learning (or model selection): The discrete computational process of transitioning from one generative model structure \(m_i\) to another \(m_j\). This occurs when the current model \(m_i\) consistently generates large prediction errors that cannot be resolved via gradient descent on \(\phi\). The controller selects a new model structure \(m_j\) from a space of possible structures \(\mathcal{M}_{space}\) such that \(\mathcal{F}(m_j) < \mathcal{F}(m_i)\). Unlike learning, this is not a continuous gradient descent, but a discrete state transition in model space.

Structure generation (or abductive generation): The stochastic or combinatorial process that dynamically creates a novel discrete generative model structure \(m_{new}\) and introduces it into the space of possible structures \(\mathcal{M}_{space}\). It is driven by the destabilizing effects of persistently high free energy when continuous learning on \(\phi\) fails. It involves altering the dimensionality of the state space \(\Omega_s\) or creating novel conditional dependencies between existing high-level representational state variables.

Space of possible structures, \(\mathcal{M}_{space}\): Rather than a static, pre-defined set of all conceivable models, \(\mathcal{M}_{space}\) should be defined as a dynamic, computationally bounded horizon of adjacent possible structures. It is continuously expanded via structure generation and pruned via model selection.

Abductive inference: The complete, discrete macro-process of creativity and paradigm shifting. It encompasses rapid structure generation of a novel hypothesis \(m_{new}\) and model selection based on the minimization of model free energy \(\mathcal{F}(m)\).

Phylogenetic structure learning (natural selection): The ultimate, population-level mechanism of model selection. Over evolutionary time, the physical environment evaluates the model evidence \(p(o \mid m)\) of an entire generation of organisms. If an organism’s endowed generative model structure \(m\) and structural setpoint distribution \(\tilde p_{struct}(s)\) are poorly calibrated to the environment, it dies. This failure effectively prunes that specific structure from the space of possible structures \(\mathcal{M}_{space}\) of the species, ensuring that only viable model structures propagate to the next generation.

Phylogenetic structure generation (mutation/recombination): The biological counterpart to abductive generation. The stochastic alteration of an organism’s morphological blueprint (DNA) during reproduction. This injects noise at the phylogenetic level, continuously populating the species’ space of possible structures \(\mathcal{M}_{space}\) with novel generative model structures \(m_{new}\) and shifted structural setpoint distributions \(\tilde p_{struct}(s)\) to be evaluated by the environment.

---

The table below summarizes the mapping between the terms above and conventional AIF terms.

Term	AIF equivalent	Role
Controller	Agent; brain	The control system controlling the controlled system by minimizing free energy.
Controlled system	Generative process	The physical system being controlled.
Physical state	External state	The state of the controlled system.
Controller state; information state	Hidden state; latent variable	The controller’s “guess” at the system state.
Viable manifold	Attracting set	The “safe” operating envelope of physical states.
Setpoint distribution	Prior; prior distribution; prior preferences; \(C\)-matrix	The setpoint or epistemic bias for the controller.
Decision	Action; control variable; \(u\)	Determines the probabilities for the state transitions in transition model calculations.
Policy	Policy	Sequence of decisions
Waypoint inference	Discrete-time POMDP; planning; policy inference	Produces the trajectory of state distributions for reaching the setpoint distribution.
Deontic model	Prior over policies; habits; \(E\)-matrix	Prior preferences over policies such as habits or moral rules.
Normative generative model	Target joint distribution / Generative model (when applied to priors)	Defines target regions and expected sensory observations to act as the mathematical baseline for free energy minimization.
Predictive generative model	Empirical generative model / Tree-search model / Instantiated generative model	Simulates prospective state sequences over a planning horizon and infers past states based on transition physics.

Artificial intelligence

Artificial intelligence (AI): The capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. “AI” is often used informally as a noun denoting software components with artificial intelligence.

Machine learning: A field of study in artificial intelligence concerned with the development and study of statistical algorithms that can learn from data and generalise to unseen data and thus perform tasks without explicit instructions.

Algorithm: A computational architecture or procedural method that can be trained on data to perform a task. Examples: convolutional neural network, transformer.

Training algorithm: A set of instructions used in machine learning to iteratively adjust the parameters of a model so it can learn from data and make accurate predictions on new data.

Model: A specific, trained AI algorithm. Example: GPT-4o.

Inference engine: A software and/or hardware component that evaluates a trained model on new data and produces output results (e.g., predictions). Example: TensorFlow.

AI application: A system in which one or more critical or high-value functions are realized by an AI component. Examples: ChatGPT, GitHub Copilot, autonomous vehicle

Systems engineering

System: An arrangement of interacting elements that together realize functions. Examples include information systems, biological organisms, ant hills, medical devices, medical laboratories, and space stations.

A system can be decomposed into (sub)systems that in turn can be decomposed into even lower-level systems, components, and ultimately parts. The term can in a specific context refer to systems as-designed (system “blueprints”), systems as-built (constructed, manufactured system), and as-maintained (operational, real-world systems).

System element: System, subsystem, component, or part.

Subsystem: A system that is part of a larger system. This term is only used informally.

Component: A system element that encapsulates a cohesive responsibility, realized by a minimal sufficient set of functions, and that interacts with its environment via a well-specified contract (its provided and required interfaces). If the contract is preserved, the component is substitutable.

Part: The lowest level of system elements for which configuration information is defined.

Risk management

Harm: Physical injury or damage to the health of people, or damage to property or the
environment.

Hazard: Potential source of harm.

Hazardous situation: Circumstance in which people, property, or the environment are
exposed to one or more hazard(s).

Risk: Combination (usually multiplication) of the probability of occurrence of harm and the
severity of that harm. Reduction of risk is the objective of risk management. A risk can be
reduced either by lowering the severity of the harm or the probability of the harm, or both.
The severity depends on the nature of the hazardous situation.

Risk control: Process in which decisions are made and measures implemented by which
risks are reduced to or maintained within specified levels. Risk control measures are also
called risk mitigations.

Fault: Condition or defect in a system which may lead to an error. Synonym: defect, bug.

Error: Manifestation of a fault as an unexpected or unwanted behavior of a system. An error
may lead to a failure.

Failure: Situation in which a system (or part of a system) is not performing its intended
function due to an error. The failure is characterized by its failure mode i.e. the specific
manner in which the failure occurs. A failure is a symptom of an error that may or may or
may not lead to a hazard.

Failure mode: The specific manner or way in which a failure occurs.

Configuration management

Configuration item (CI): A set of functional, performance, and physical characteristics of a system element; a set of true statements about a system element that we choose to manage as a unit. This is the most important term in configuration management.

A configuration item has an extension in time. It comes into existence at one point in time, e.g., when it is defined in the systems engineering process, and it goes out of existence, e.g., when the whole system reaches end of life or when the configuration item is replaced by a new configuration item because of a new system architecture.

Configuration items usually evolve over time as the system is improved (except as-built configuration items, see below). The evolution of the configuration item is documented as a sequence of configuration item versions, each of which defines the configuration information during a defined period of time, the validity period.

Configuration item versions are in turn documented in the *configuration information*, a set of documents, models etc, associated with the configuration item (see Figure 1a and Figure 3).

In the most general case, configuration management spans over the following types of configuration items:

As-designed configurations item: (Predicted) characteristics of designed (but not necessarily yet constructed) system element.

As-built configurations items: (Actual and predicted) characteristics of constructed (but not yet put in operation) system element.

As-maintained configuration item: (Actual) characteristics of a system element in operations.

A configuration item should be selected so that it is realized by one system or system element. Configuration items should furthermore be selected so that they can be managed with minimal dependency on other configuration items (modularity). Configuration item selection criteria should therefore consider:

• Regulatory requirements
• Criticality in terms of risks and safety
• Anticipation of new technology replacing old
• Interfaces with other configuration items
• Procurement
• Support and service

Examples:

• Components that are procured as a single entity (e.g., an x-ray tube) should be represented by one configuration item that defines the specifications of the component.
• System elements that have their own maintenance schedule and maintenance log should be represented by one configuration item that is all the maintenance-related information.
• System elements that are developed as one entity, with their own specifications, risk analyses etc. should map to one configuration item.

Configuration: A top-level (root) configuration item; the characteristics of a system element that is managed as an independent entity, often a whole system.

Configuration item version (CIV): A (long exposure time) snapshot of a configuration item between two timestamps, `valid_from` and `valid_to`.

If `valid_to` is not set or set to a future date, then the configuration item is currently valid.

For as-designed configuration items valid means released, i.e., formally approved for construction after `valid_from`.

If the last version of an as-designed configuration item has its `valid_to` in the past then it is deleted from the configuration, i.e., the corresponding system element is no longer part of the system.

As built configuration items are kept valid for as long as the system is assumed to be on the market or, in case of medical devices, sometimes indefinitely.

For as-maintained configuration items valid means installed, i.e., operational in a system instance.

An as-maintained configuration item with its `valid_to` in the past is removed from the operational system.

Key configuration item versions (baselines) can also be given a human-readable label, like v2.1.0.

Configuration version: A snapshot of a configuration at a well-defined point in time; a top-level configuration item version.

Configuration baseline: A formally reviewed and approved configuration version that serves as a non-volatile reference point for a well-defined purpose.

Configuration item graph: A directed acyclic graph (DAG) of configuration items. The graph mirrors the structural decomposition of the system of interest. The reason that the CI graph is not a simple tree is that the same configuration item (e.g., a power outlet) may appear in many branches of the (as-built) configuration item graph in an as-designed configuration. The “acyclic” adjective means that a configuration item can not be part of itself; there can not be any circular part-of associations.

An example of a CI graph is shown in the figure below. Note that Part 3 is part of both Component 1 and Component 2.

Configuration information: A representation of a configuration or a configuration item. Typically a set of documents, source code, models, database records etc that together document the characteristics of a system element (a configuration item).

Information item: A repository of configuration information such as a document, a model, a source code file, or a database record. Information items evolve over time. Their state at a certain point in time is represented by an information item version, a snapshot of the information item at that point in time.

Information item version (IIV): A snapshot of an information item at a well-defined point in time.

Change request: A formal request to change a baseline, e.g., by adding a new function to the system. The change request is not part of the configuration information since it describes a difference between two configuration baselines rather than a configuration baseline.