Time representation in reinforcement learning models of the basal ganglia
Reinforcement learning (RL) models have been influential in understanding many aspects of basal ganglia function, from reward prediction to action selection. Time plays an important role in these models, but there is still no theoretical consensus about what kind of time representation is used by th...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Frontiers Research Foundation
2015
|
| Online Access: | http://hdl.handle.net/1721.1/94333 |
| _version_ | 1826212589423558656 |
|---|---|
| author | Gershman, Samuel J. Moustafa, Ahmed A. Ludvig, Elliot A. |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Gershman, Samuel J. Moustafa, Ahmed A. Ludvig, Elliot A. |
| author_sort | Gershman, Samuel J. |
| collection | MIT |
| description | Reinforcement learning (RL) models have been influential in understanding many aspects of basal ganglia function, from reward prediction to action selection. Time plays an important role in these models, but there is still no theoretical consensus about what kind of time representation is used by the basal ganglia. We review several theoretical accounts and their supporting evidence. We then discuss the relationship between RL models and the timing mechanisms that have been attributed to the basal ganglia. We hypothesize that a single computational system may underlie both RL and interval timing—the perception of duration in the range of seconds to hours. This hypothesis, which extends earlier models by incorporating a time-sensitive action selection mechanism, may have important implications for understanding disorders like Parkinson's disease in which both decision making and timing are impaired. |
| first_indexed | 2024-09-23T15:26:43Z |
| format | Article |
| id | mit-1721.1/94333 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:26:43Z |
| publishDate | 2015 |
| publisher | Frontiers Research Foundation |
| record_format | dspace |
| spelling | mit-1721.1/943332022-09-29T14:42:36Z Time representation in reinforcement learning models of the basal ganglia Gershman, Samuel J. Moustafa, Ahmed A. Ludvig, Elliot A. Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Gershman, Samuel J. Reinforcement learning (RL) models have been influential in understanding many aspects of basal ganglia function, from reward prediction to action selection. Time plays an important role in these models, but there is still no theoretical consensus about what kind of time representation is used by the basal ganglia. We review several theoretical accounts and their supporting evidence. We then discuss the relationship between RL models and the timing mechanisms that have been attributed to the basal ganglia. We hypothesize that a single computational system may underlie both RL and interval timing—the perception of duration in the range of seconds to hours. This hypothesis, which extends earlier models by incorporating a time-sensitive action selection mechanism, may have important implications for understanding disorders like Parkinson's disease in which both decision making and timing are impaired. United States. Intelligence Advanced Research Projects Activity (Contract D10PC2002) MIT Intelligence Initiative (Postdoctoral Fellowship) 2015-02-11T17:58:50Z 2015-02-11T17:58:50Z 2014-01 2013-10 Article http://purl.org/eprint/type/JournalArticle 1662-5188 http://hdl.handle.net/1721.1/94333 Gershman, Samuel J., Ahmed A. Moustafa, and Elliot A. Ludvig. “Time Representation in Reinforcement Learning Models of the Basal Ganglia.” Frontiers in Computational Neuroscience 7 (2014). en_US http://dx.doi.org/10.3389/fncom.2013.00194 Frontiers in Computational Neuroscience Creative Commons Attribution http://creativecommons.org/licenses/by/4.0/ application/pdf Frontiers Research Foundation Frontiers Research Foundation |
| spellingShingle | Gershman, Samuel J. Moustafa, Ahmed A. Ludvig, Elliot A. Time representation in reinforcement learning models of the basal ganglia |
| title | Time representation in reinforcement learning models of the basal ganglia |
| title_full | Time representation in reinforcement learning models of the basal ganglia |
| title_fullStr | Time representation in reinforcement learning models of the basal ganglia |
| title_full_unstemmed | Time representation in reinforcement learning models of the basal ganglia |
| title_short | Time representation in reinforcement learning models of the basal ganglia |
| title_sort | time representation in reinforcement learning models of the basal ganglia |
| url | http://hdl.handle.net/1721.1/94333 |
| work_keys_str_mv | AT gershmansamuelj timerepresentationinreinforcementlearningmodelsofthebasalganglia AT moustafaahmeda timerepresentationinreinforcementlearningmodelsofthebasalganglia AT ludvigelliota timerepresentationinreinforcementlearningmodelsofthebasalganglia |