Dopamine, uncertainty and TD learning
<p>Abstract</p> <p>Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference (TD) error in predictions of future reward, with increases above and decreases below baseline consequent on positive and negat...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2005-05-01
|
| Series: | Behavioral and Brain Functions |
| Online Access: | http://www.behavioralandbrainfunctions.com/content/1/1/6 |
| _version_ | 1818807406793064448 |
|---|---|
| author | Duff Michael O Niv Yael Dayan Peter |
| author_facet | Duff Michael O Niv Yael Dayan Peter |
| author_sort | Duff Michael O |
| collection | DOAJ |
| description | <p>Abstract</p> <p>Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference (TD) error in predictions of future reward, with increases above and decreases below baseline consequent on positive and negative prediction errors, respectively. However, dopamine cells have very low baseline activity, which implies that the representation of these two sorts of error is asymmetric. We explore the implications of this seemingly innocuous asymmetry for the interpretation of dopaminergic firing patterns in experiments with probabilistic rewards which bring about persistent prediction errors. In particular, we show that when averaging the non-stationary prediction errors across trials, a ramping in the activity of the dopamine neurons should be apparent, whose magnitude is dependent on the learning rate. This exact phenomenon was observed in a recent experiment, though being interpreted there in antipodal terms as a within-trial encoding of uncertainty.</p> |
| first_indexed | 2024-12-18T19:25:10Z |
| format | Article |
| id | doaj.art-ebd778ddcde244779aa45b8a13514ebc |
| institution | Directory Open Access Journal |
| issn | 1744-9081 |
| language | English |
| last_indexed | 2024-12-18T19:25:10Z |
| publishDate | 2005-05-01 |
| publisher | BMC |
| record_format | Article |
| series | Behavioral and Brain Functions |
| spelling | doaj.art-ebd778ddcde244779aa45b8a13514ebc2022-12-21T20:55:53ZengBMCBehavioral and Brain Functions1744-90812005-05-0111610.1186/1744-9081-1-6Dopamine, uncertainty and TD learningDuff Michael ONiv YaelDayan Peter<p>Abstract</p> <p>Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference (TD) error in predictions of future reward, with increases above and decreases below baseline consequent on positive and negative prediction errors, respectively. However, dopamine cells have very low baseline activity, which implies that the representation of these two sorts of error is asymmetric. We explore the implications of this seemingly innocuous asymmetry for the interpretation of dopaminergic firing patterns in experiments with probabilistic rewards which bring about persistent prediction errors. In particular, we show that when averaging the non-stationary prediction errors across trials, a ramping in the activity of the dopamine neurons should be apparent, whose magnitude is dependent on the learning rate. This exact phenomenon was observed in a recent experiment, though being interpreted there in antipodal terms as a within-trial encoding of uncertainty.</p>http://www.behavioralandbrainfunctions.com/content/1/1/6 |
| spellingShingle | Duff Michael O Niv Yael Dayan Peter Dopamine, uncertainty and TD learning Behavioral and Brain Functions |
| title | Dopamine, uncertainty and TD learning |
| title_full | Dopamine, uncertainty and TD learning |
| title_fullStr | Dopamine, uncertainty and TD learning |
| title_full_unstemmed | Dopamine, uncertainty and TD learning |
| title_short | Dopamine, uncertainty and TD learning |
| title_sort | dopamine uncertainty and td learning |
| url | http://www.behavioralandbrainfunctions.com/content/1/1/6 |
| work_keys_str_mv | AT duffmichaelo dopamineuncertaintyandtdlearning AT nivyael dopamineuncertaintyandtdlearning AT dayanpeter dopamineuncertaintyandtdlearning |