A hypothesis for basal ganglia-dependent reinforcement learning in the songbird
Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence ca...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Elsevier BV
2020
|
| Online Access: | https://hdl.handle.net/1721.1/126651 |
| _version_ | 1826195942267682816 |
|---|---|
| author | Fee, Michale S. Goldberg, Jesse H. |
| author2 | McGovern Institute for Brain Research at MIT |
| author_facet | McGovern Institute for Brain Research at MIT Fee, Michale S. Goldberg, Jesse H. |
| author_sort | Fee, Michale S. |
| collection | MIT |
| description | Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in ''biasing'' this variability to improve behavioral performance. This BG-dependent ''premotor bias'' may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.This article is part of a Special Issue entitled: Function and Dysfunction of the Basal Ganglia. |
| first_indexed | 2024-09-23T10:18:16Z |
| format | Article |
| id | mit-1721.1/126651 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T10:18:16Z |
| publishDate | 2020 |
| publisher | Elsevier BV |
| record_format | dspace |
| spelling | mit-1721.1/1266512022-09-26T17:07:00Z A hypothesis for basal ganglia-dependent reinforcement learning in the songbird Fee, Michale S. Goldberg, Jesse H. McGovern Institute for Brain Research at MIT Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in ''biasing'' this variability to improve behavioral performance. This BG-dependent ''premotor bias'' may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.This article is part of a Special Issue entitled: Function and Dysfunction of the Basal Ganglia. NIH (Grant R01DC009183, R01MH067105 and K99NS067062) 2020-08-18T17:52:51Z 2020-08-18T17:52:51Z 2011-12 2019-09-30T18:09:22Z Article http://purl.org/eprint/type/JournalArticle 0306-4522 https://hdl.handle.net/1721.1/126651 Fee, M.S. and J. H. Goldberg. "A hypothesis for basal ganglia-dependent reinforcement learning in the songbird." Neuroscience 198 (December 2011): 152-170 © 2011 IBRO en http://dx.doi.org/10.1016/j.neuroscience.2011.09.069 Neuroscience Creative Commons Attribution-NonCommercial-NoDerivs License http://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Elsevier BV PMC |
| spellingShingle | Fee, Michale S. Goldberg, Jesse H. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird |
| title | A hypothesis for basal ganglia-dependent reinforcement learning in the songbird |
| title_full | A hypothesis for basal ganglia-dependent reinforcement learning in the songbird |
| title_fullStr | A hypothesis for basal ganglia-dependent reinforcement learning in the songbird |
| title_full_unstemmed | A hypothesis for basal ganglia-dependent reinforcement learning in the songbird |
| title_short | A hypothesis for basal ganglia-dependent reinforcement learning in the songbird |
| title_sort | hypothesis for basal ganglia dependent reinforcement learning in the songbird |
| url | https://hdl.handle.net/1721.1/126651 |
| work_keys_str_mv | AT feemichales ahypothesisforbasalgangliadependentreinforcementlearninginthesongbird AT goldbergjesseh ahypothesisforbasalgangliadependentreinforcementlearninginthesongbird AT feemichales hypothesisforbasalgangliadependentreinforcementlearninginthesongbird AT goldbergjesseh hypothesisforbasalgangliadependentreinforcementlearninginthesongbird |