Active inference and the two-step task
Abstract Sequential decision problems distill important challenges frequently faced by humans. Through repeated interactions with an uncertain world, unknown statistics need to be learned while balancing exploration and exploitation. Reinforcement learning is a prominent method for modeling such beh...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-10-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-21766-4 |
| _version_ | 1798028624521068544 |
|---|---|
| author | Sam Gijsen Miro Grundei Felix Blankenburg |
| author_facet | Sam Gijsen Miro Grundei Felix Blankenburg |
| author_sort | Sam Gijsen |
| collection | DOAJ |
| description | Abstract Sequential decision problems distill important challenges frequently faced by humans. Through repeated interactions with an uncertain world, unknown statistics need to be learned while balancing exploration and exploitation. Reinforcement learning is a prominent method for modeling such behaviour, with a prevalent application being the two-step task. However, recent studies indicate that the standard reinforcement learning model sometimes describes features of human task behaviour inaccurately and incompletely. We investigated whether active inference, a framework proposing a trade-off to the exploration-exploitation dilemma, could better describe human behaviour. Therefore, we re-analysed four publicly available datasets of the two-step task, performed Bayesian model selection, and compared behavioural model predictions. Two datasets, which revealed more model-based inference and behaviour indicative of directed exploration, were better described by active inference, while the models scored similarly for the remaining datasets. Learning using probability distributions appears to contribute to the improved model fits. Further, approximately half of all participants showed sensitivity to information gain as formulated under active inference, although behavioural exploration effects were not fully captured. These results contribute to the empirical validation of active inference as a model of human behaviour and the study of alternative models for the influential two-step task. |
| first_indexed | 2024-04-11T19:10:56Z |
| format | Article |
| id | doaj.art-997db051059c489f8d57b7c3ecdabef8 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-11T19:10:56Z |
| publishDate | 2022-10-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-997db051059c489f8d57b7c3ecdabef82022-12-22T04:07:37ZengNature PortfolioScientific Reports2045-23222022-10-0112111510.1038/s41598-022-21766-4Active inference and the two-step taskSam Gijsen0Miro Grundei1Felix Blankenburg2Neurocomputation and Neuroimaging Unit, Freie Universität BerlinNeurocomputation and Neuroimaging Unit, Freie Universität BerlinNeurocomputation and Neuroimaging Unit, Freie Universität BerlinAbstract Sequential decision problems distill important challenges frequently faced by humans. Through repeated interactions with an uncertain world, unknown statistics need to be learned while balancing exploration and exploitation. Reinforcement learning is a prominent method for modeling such behaviour, with a prevalent application being the two-step task. However, recent studies indicate that the standard reinforcement learning model sometimes describes features of human task behaviour inaccurately and incompletely. We investigated whether active inference, a framework proposing a trade-off to the exploration-exploitation dilemma, could better describe human behaviour. Therefore, we re-analysed four publicly available datasets of the two-step task, performed Bayesian model selection, and compared behavioural model predictions. Two datasets, which revealed more model-based inference and behaviour indicative of directed exploration, were better described by active inference, while the models scored similarly for the remaining datasets. Learning using probability distributions appears to contribute to the improved model fits. Further, approximately half of all participants showed sensitivity to information gain as formulated under active inference, although behavioural exploration effects were not fully captured. These results contribute to the empirical validation of active inference as a model of human behaviour and the study of alternative models for the influential two-step task.https://doi.org/10.1038/s41598-022-21766-4 |
| spellingShingle | Sam Gijsen Miro Grundei Felix Blankenburg Active inference and the two-step task Scientific Reports |
| title | Active inference and the two-step task |
| title_full | Active inference and the two-step task |
| title_fullStr | Active inference and the two-step task |
| title_full_unstemmed | Active inference and the two-step task |
| title_short | Active inference and the two-step task |
| title_sort | active inference and the two step task |
| url | https://doi.org/10.1038/s41598-022-21766-4 |
| work_keys_str_mv | AT samgijsen activeinferenceandthetwosteptask AT mirogrundei activeinferenceandthetwosteptask AT felixblankenburg activeinferenceandthetwosteptask |