Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity
The brain enables animals to behaviourally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviours are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal gang...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2016-07-01
|
| Series: | Frontiers in Neural Circuits |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fncir.2016.00053/full |
| _version_ | 1819015608362074112 |
|---|---|
| author | Pierre Berthet Pierre Berthet Pierre Berthet Mikael Lindahl Mikael Lindahl Philip Joseph Tully Philip Joseph Tully Philip Joseph Tully Jeanette Hellgren Kotaleski Jeanette Hellgren Kotaleski Jeanette Hellgren Kotaleski Anders Lansner Anders Lansner Anders Lansner |
| author_facet | Pierre Berthet Pierre Berthet Pierre Berthet Mikael Lindahl Mikael Lindahl Philip Joseph Tully Philip Joseph Tully Philip Joseph Tully Jeanette Hellgren Kotaleski Jeanette Hellgren Kotaleski Jeanette Hellgren Kotaleski Anders Lansner Anders Lansner Anders Lansner |
| author_sort | Pierre Berthet |
| collection | DOAJ |
| description | The brain enables animals to behaviourally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviours are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and postsynaptic activity, receptor type and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson’s disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1. |
| first_indexed | 2024-12-21T02:34:26Z |
| format | Article |
| id | doaj.art-a97a7d7becd944a78465d2d4e9a3d9a8 |
| institution | Directory Open Access Journal |
| issn | 1662-5110 |
| language | English |
| last_indexed | 2024-12-21T02:34:26Z |
| publishDate | 2016-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neural Circuits |
| spelling | doaj.art-a97a7d7becd944a78465d2d4e9a3d9a82022-12-21T19:18:50ZengFrontiers Media S.A.Frontiers in Neural Circuits1662-51102016-07-011010.3389/fncir.2016.00053180659Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent PlasticityPierre Berthet0Pierre Berthet1Pierre Berthet2Mikael Lindahl3Mikael Lindahl4Philip Joseph Tully5Philip Joseph Tully6Philip Joseph Tully7Jeanette Hellgren Kotaleski8Jeanette Hellgren Kotaleski9Jeanette Hellgren Kotaleski10Anders Lansner11Anders Lansner12Anders Lansner13Stockholms UniversitetKTH Royal Institute of TechnologyStockholm Brain InstituteKTH Royal Institute of TechnologyStockholm Brain InstituteKTH Royal Institute of TechnologyStockholm Brain InstituteUniversity of EdinburghKTH Royal Institute of TechnologyStockholm Brain InstituteKarolinska InstituteStockholms UniversitetKTH Royal Institute of TechnologyStockholm Brain InstituteThe brain enables animals to behaviourally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviours are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and postsynaptic activity, receptor type and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson’s disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.http://journal.frontiersin.org/Journal/10.3389/fncir.2016.00053/fullBasal GangliaDopamineParkinson Diseasereinforcement learningsynaptic plasticityaction selection |
| spellingShingle | Pierre Berthet Pierre Berthet Pierre Berthet Mikael Lindahl Mikael Lindahl Philip Joseph Tully Philip Joseph Tully Philip Joseph Tully Jeanette Hellgren Kotaleski Jeanette Hellgren Kotaleski Jeanette Hellgren Kotaleski Anders Lansner Anders Lansner Anders Lansner Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity Frontiers in Neural Circuits Basal Ganglia Dopamine Parkinson Disease reinforcement learning synaptic plasticity action selection |
| title | Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity |
| title_full | Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity |
| title_fullStr | Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity |
| title_full_unstemmed | Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity |
| title_short | Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity |
| title_sort | functional relevance of different basal ganglia pathways investigated in a spiking model with reward dependent plasticity |
| topic | Basal Ganglia Dopamine Parkinson Disease reinforcement learning synaptic plasticity action selection |
| url | http://journal.frontiersin.org/Journal/10.3389/fncir.2016.00053/full |
| work_keys_str_mv | AT pierreberthet functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT pierreberthet functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT pierreberthet functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT mikaellindahl functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT mikaellindahl functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT philipjosephtully functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT philipjosephtully functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT philipjosephtully functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT jeanettehellgrenkotaleski functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT jeanettehellgrenkotaleski functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT jeanettehellgrenkotaleski functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT anderslansner functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT anderslansner functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity AT anderslansner functionalrelevanceofdifferentbasalgangliapathwaysinvestigatedinaspikingmodelwithrewarddependentplasticity |