Integration of reinforcement learning and optimal decision-making theories of the basal ganglia.
This article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory...
| Main Authors: | , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
2011
|
| _version_ | 1826280419582017536 |
|---|---|
| author | Bogacz, R Larsen, T |
| author_facet | Bogacz, R Larsen, T |
| author_sort | Bogacz, R |
| collection | OXFORD |
| description | This article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory evidence accumulated in the cortex. In particular, we present a model that integrates the actor-critic model of reinforcement learning and a model assuming that the cortico-basal-ganglia circuit implements a statistically optimal decision-making procedure. The values of cortico-striatal weights required for optimal decision making in our model differ from those provided by standard reinforcement learning models. Nevertheless, we show that an actor-critic model converges to the weights required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predictions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforcement learning and optimal decision-making theories. |
| first_indexed | 2024-03-07T00:13:26Z |
| format | Journal article |
| id | oxford-uuid:7a069246-74d5-48de-819f-1e637e6cdfcc |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T00:13:26Z |
| publishDate | 2011 |
| record_format | dspace |
| spelling | oxford-uuid:7a069246-74d5-48de-819f-1e637e6cdfcc2022-03-26T20:41:11ZIntegration of reinforcement learning and optimal decision-making theories of the basal ganglia.Journal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:7a069246-74d5-48de-819f-1e637e6cdfccEnglishSymplectic Elements at Oxford2011Bogacz, RLarsen, TThis article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory evidence accumulated in the cortex. In particular, we present a model that integrates the actor-critic model of reinforcement learning and a model assuming that the cortico-basal-ganglia circuit implements a statistically optimal decision-making procedure. The values of cortico-striatal weights required for optimal decision making in our model differ from those provided by standard reinforcement learning models. Nevertheless, we show that an actor-critic model converges to the weights required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predictions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforcement learning and optimal decision-making theories. |
| spellingShingle | Bogacz, R Larsen, T Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. |
| title | Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. |
| title_full | Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. |
| title_fullStr | Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. |
| title_full_unstemmed | Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. |
| title_short | Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. |
| title_sort | integration of reinforcement learning and optimal decision making theories of the basal ganglia |
| work_keys_str_mv | AT bogaczr integrationofreinforcementlearningandoptimaldecisionmakingtheoriesofthebasalganglia AT larsent integrationofreinforcementlearningandoptimaldecisionmakingtheoriesofthebasalganglia |