On Epistemics in Expected Free Energy for Linear Gaussian State Space Models
Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the fr...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/12/1565 |
| _version_ | 1797504961331855360 |
|---|---|
| author | Magnus T. Koudahl Wouter M. Kouw Bert de Vries |
| author_facet | Magnus T. Koudahl Wouter M. Kouw Bert de Vries |
| author_sort | Magnus T. Koudahl |
| collection | DOAJ |
| description | Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive. |
| first_indexed | 2024-03-10T04:11:51Z |
| format | Article |
| id | doaj.art-2accd9315e9d4561b6b489068d39fa18 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T04:11:51Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-2accd9315e9d4561b6b489068d39fa182023-11-23T08:09:59ZengMDPI AGEntropy1099-43002021-11-012312156510.3390/e23121565On Epistemics in Expected Free Energy for Linear Gaussian State Space ModelsMagnus T. Koudahl0Wouter M. Kouw1Bert de Vries2Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The NetherlandsDepartment of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The NetherlandsDepartment of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The NetherlandsActive Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.https://www.mdpi.com/1099-4300/23/12/1565active inferenceepistemicsexpected free energyfree energy principlelinear Gaussian dynamical system |
| spellingShingle | Magnus T. Koudahl Wouter M. Kouw Bert de Vries On Epistemics in Expected Free Energy for Linear Gaussian State Space Models Entropy active inference epistemics expected free energy free energy principle linear Gaussian dynamical system |
| title | On Epistemics in Expected Free Energy for Linear Gaussian State Space Models |
| title_full | On Epistemics in Expected Free Energy for Linear Gaussian State Space Models |
| title_fullStr | On Epistemics in Expected Free Energy for Linear Gaussian State Space Models |
| title_full_unstemmed | On Epistemics in Expected Free Energy for Linear Gaussian State Space Models |
| title_short | On Epistemics in Expected Free Energy for Linear Gaussian State Space Models |
| title_sort | on epistemics in expected free energy for linear gaussian state space models |
| topic | active inference epistemics expected free energy free energy principle linear Gaussian dynamical system |
| url | https://www.mdpi.com/1099-4300/23/12/1565 |
| work_keys_str_mv | AT magnustkoudahl onepistemicsinexpectedfreeenergyforlineargaussianstatespacemodels AT woutermkouw onepistemicsinexpectedfreeenergyforlineargaussianstatespacemodels AT bertdevries onepistemicsinexpectedfreeenergyforlineargaussianstatespacemodels |