Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations
Dopamine (DA) neurons in the ventral tegmental area (VTA) are thought to encode reward prediction errors (RPE) by comparing actual and expected rewards. In recent years, much work has been done to identify how the brain uses and computes this signal. While several lines of evidence suggest the inter...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2019-01-01
|
| Series: | Frontiers in Neural Circuits |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fncir.2018.00116/full |
| _version_ | 1818777239871815680 |
|---|---|
| author | Nicolas Deperrois Victoria Moiseeva Boris Gutkin Boris Gutkin |
| author_facet | Nicolas Deperrois Victoria Moiseeva Boris Gutkin Boris Gutkin |
| author_sort | Nicolas Deperrois |
| collection | DOAJ |
| description | Dopamine (DA) neurons in the ventral tegmental area (VTA) are thought to encode reward prediction errors (RPE) by comparing actual and expected rewards. In recent years, much work has been done to identify how the brain uses and computes this signal. While several lines of evidence suggest the interplay of the DA and the inhibitory interneurons in the VTA implements the RPE computation, it still remains unclear how the DA neurons learn key quantities, for example the amplitude and the timing of primary rewards during conditioning tasks. Furthermore, endogenous acetylcholine and exogenous nicotine, also likely affect these computations by acting on both VTA DA and GABA (γ -aminobutyric acid) neurons via nicotinic-acetylcholine receptors (nAChRs). To explore the potential circuit-level mechanisms for RPE computations during classical-conditioning tasks, we developed a minimal computational model of the VTA circuitry. The model was designed to account for several reward-related properties of VTA afferents and recent findings on VTA GABA neuron dynamics during conditioning. With our minimal model, we showed that the RPE can be learned by a two-speed process computing reward timing and magnitude. By including models of nAChR-mediated currents in the VTA DA-GABA circuit, we showed that nicotine should reduce the acetylcholine action on the VTA GABA neurons by receptor desensitization and potentially boost DA responses to reward-related signals in a non-trivial manner. Together, our results delineate the mechanisms by which RPE are computed in the brain, and suggest a hypothesis on nicotine-mediated effects on reward-related perception and decision-making. |
| first_indexed | 2024-12-18T11:25:40Z |
| format | Article |
| id | doaj.art-a284fd6e199b4229b2d4ddcddb2ee332 |
| institution | Directory Open Access Journal |
| issn | 1662-5110 |
| language | English |
| last_indexed | 2024-12-18T11:25:40Z |
| publishDate | 2019-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neural Circuits |
| spelling | doaj.art-a284fd6e199b4229b2d4ddcddb2ee3322022-12-21T21:09:42ZengFrontiers Media S.A.Frontiers in Neural Circuits1662-51102019-01-011210.3389/fncir.2018.00116425024Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic ModulationsNicolas Deperrois0Victoria Moiseeva1Boris Gutkin2Boris Gutkin3Group for Neural Theory, LNC2 INSERM U960, DEC, École Normale Supérieure PSL* University, Paris, FranceCenter for Cognition and Decision Making, Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, RussiaGroup for Neural Theory, LNC2 INSERM U960, DEC, École Normale Supérieure PSL* University, Paris, FranceCenter for Cognition and Decision Making, Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, RussiaDopamine (DA) neurons in the ventral tegmental area (VTA) are thought to encode reward prediction errors (RPE) by comparing actual and expected rewards. In recent years, much work has been done to identify how the brain uses and computes this signal. While several lines of evidence suggest the interplay of the DA and the inhibitory interneurons in the VTA implements the RPE computation, it still remains unclear how the DA neurons learn key quantities, for example the amplitude and the timing of primary rewards during conditioning tasks. Furthermore, endogenous acetylcholine and exogenous nicotine, also likely affect these computations by acting on both VTA DA and GABA (γ -aminobutyric acid) neurons via nicotinic-acetylcholine receptors (nAChRs). To explore the potential circuit-level mechanisms for RPE computations during classical-conditioning tasks, we developed a minimal computational model of the VTA circuitry. The model was designed to account for several reward-related properties of VTA afferents and recent findings on VTA GABA neuron dynamics during conditioning. With our minimal model, we showed that the RPE can be learned by a two-speed process computing reward timing and magnitude. By including models of nAChR-mediated currents in the VTA DA-GABA circuit, we showed that nicotine should reduce the acetylcholine action on the VTA GABA neurons by receptor desensitization and potentially boost DA responses to reward-related signals in a non-trivial manner. Together, our results delineate the mechanisms by which RPE are computed in the brain, and suggest a hypothesis on nicotine-mediated effects on reward-related perception and decision-making.https://www.frontiersin.org/article/10.3389/fncir.2018.00116/fulldopaminereward-prediction errorventral tegmental areaacetylcholinenicotine |
| spellingShingle | Nicolas Deperrois Victoria Moiseeva Boris Gutkin Boris Gutkin Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations Frontiers in Neural Circuits dopamine reward-prediction error ventral tegmental area acetylcholine nicotine |
| title | Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations |
| title_full | Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations |
| title_fullStr | Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations |
| title_full_unstemmed | Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations |
| title_short | Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations |
| title_sort | minimal circuit model of reward prediction error computations and effects of nicotinic modulations |
| topic | dopamine reward-prediction error ventral tegmental area acetylcholine nicotine |
| url | https://www.frontiersin.org/article/10.3389/fncir.2018.00116/full |
| work_keys_str_mv | AT nicolasdeperrois minimalcircuitmodelofrewardpredictionerrorcomputationsandeffectsofnicotinicmodulations AT victoriamoiseeva minimalcircuitmodelofrewardpredictionerrorcomputationsandeffectsofnicotinicmodulations AT borisgutkin minimalcircuitmodelofrewardpredictionerrorcomputationsandeffectsofnicotinicmodulations AT borisgutkin minimalcircuitmodelofrewardpredictionerrorcomputationsandeffectsofnicotinicmodulations |