Making predictions in a changing world – inference, uncertainty and learning
To function effectively, brains need to make predictions about their environment based on past experience, i.e. they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a t...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2013-06-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fnins.2013.00105/full |
| _version_ | 1818849035396579328 |
|---|---|
| author | Jill X O'reilly |
| author_facet | Jill X O'reilly |
| author_sort | Jill X O'reilly |
| collection | DOAJ |
| description | To function effectively, brains need to make predictions about their environment based on past experience, i.e. they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a tool to model participants' incomplete knowledge of task parameters and hence, to better understand their behaviour.This review focusses on a particular challenge for learning algorithms - how to match the rate at which they learn to the rate of change in the environment, so that they use as much observed data as possible whilst disregarding irrelevant, old observations. To do this algorithms must evaluate whether the environment is changing. We discuss the concepts of likelihood, priors and transition functions, and how these relate to change detection. We review expected and estimation uncertainty, and how these relate to change detection and learning rate. Finally, we consider the neural correlates of uncertainty and learning. We argue that the neural correlates of uncertainty bear a resemblance to neural systems that are active when agents actively explore their environments, suggesting that the mechanisms by which the rate of learning is set may be subject to top down control (in circumstances when agents actively seek new information) as well as bottom up control (by observations that imply change in the environment. |
| first_indexed | 2024-12-19T06:26:50Z |
| format | Article |
| id | doaj.art-bab3efc357334d2eb18dad3e2d6fde7a |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-19T06:26:50Z |
| publishDate | 2013-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-bab3efc357334d2eb18dad3e2d6fde7a2022-12-21T20:32:31ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2013-06-01710.3389/fnins.2013.0010533773Making predictions in a changing world – inference, uncertainty and learningJill X O'reilly0University of OxfordTo function effectively, brains need to make predictions about their environment based on past experience, i.e. they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a tool to model participants' incomplete knowledge of task parameters and hence, to better understand their behaviour.This review focusses on a particular challenge for learning algorithms - how to match the rate at which they learn to the rate of change in the environment, so that they use as much observed data as possible whilst disregarding irrelevant, old observations. To do this algorithms must evaluate whether the environment is changing. We discuss the concepts of likelihood, priors and transition functions, and how these relate to change detection. We review expected and estimation uncertainty, and how these relate to change detection and learning rate. Finally, we consider the neural correlates of uncertainty and learning. We argue that the neural correlates of uncertainty bear a resemblance to neural systems that are active when agents actively explore their environments, suggesting that the mechanisms by which the rate of learning is set may be subject to top down control (in circumstances when agents actively seek new information) as well as bottom up control (by observations that imply change in the environment.http://journal.frontiersin.org/Journal/10.3389/fnins.2013.00105/fullExploratory BehaviorLearningmodellinguncertaintychange detectionBayes Theorem |
| spellingShingle | Jill X O'reilly Making predictions in a changing world – inference, uncertainty and learning Frontiers in Neuroscience Exploratory Behavior Learning modelling uncertainty change detection Bayes Theorem |
| title | Making predictions in a changing world – inference, uncertainty and learning |
| title_full | Making predictions in a changing world – inference, uncertainty and learning |
| title_fullStr | Making predictions in a changing world – inference, uncertainty and learning |
| title_full_unstemmed | Making predictions in a changing world – inference, uncertainty and learning |
| title_short | Making predictions in a changing world – inference, uncertainty and learning |
| title_sort | making predictions in a changing world inference uncertainty and learning |
| topic | Exploratory Behavior Learning modelling uncertainty change detection Bayes Theorem |
| url | http://journal.frontiersin.org/Journal/10.3389/fnins.2013.00105/full |
| work_keys_str_mv | AT jillxoreilly makingpredictionsinachangingworldinferenceuncertaintyandlearning |