Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model
Dopaminergic neurons in the mammalian substantia nigra displaycharacteristic phasic responses to stimuli which reliably predict thereceipt of primary rewards. These responses have been suggested toencode reward prediction-errors similar to those used in reinforcementlearning. Here, we propose a mod...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2011-05-01
|
| Series: | Frontiers in Computational Neuroscience |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fncom.2011.00021/full |
| _version_ | 1811225106101305344 |
|---|---|
| author | Paul eChorley Anil K Seth |
| author_facet | Paul eChorley Anil K Seth |
| author_sort | Paul eChorley |
| collection | DOAJ |
| description | Dopaminergic neurons in the mammalian substantia nigra displaycharacteristic phasic responses to stimuli which reliably predict thereceipt of primary rewards. These responses have been suggested toencode reward prediction-errors similar to those used in reinforcementlearning. Here, we propose a model of dopaminergic activity in whichprediction error signals are generated by the joint action ofshort-latency excitation and long-latency inhibition, in a networkundergoing dopaminergic neuromodulation of both spike-timing dependentsynaptic plasticity and neuronal excitability. In contrast toprevious models, sensitivity to recent events is maintained by theselective modification of specific striatal synapses, efferent tocortical neurons exhibiting stimulus-specific, temporally extendedactivity patterns. Our model shows, in the presence of significantbackground activity, (i) a shift in dopaminergic response from rewardto reward predicting stimuli, (ii) preservation of a response tounexpected rewards, and (iii) a precisely-timed below-baseline dip inactivity observed when expected rewards are omitted. |
| first_indexed | 2024-04-12T09:00:12Z |
| format | Article |
| id | doaj.art-0770f2b13315445d92b86fa54941b639 |
| institution | Directory Open Access Journal |
| issn | 1662-5188 |
| language | English |
| last_indexed | 2024-04-12T09:00:12Z |
| publishDate | 2011-05-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computational Neuroscience |
| spelling | doaj.art-0770f2b13315445d92b86fa54941b6392022-12-22T03:39:15ZengFrontiers Media S.A.Frontiers in Computational Neuroscience1662-51882011-05-01510.3389/fncom.2011.000219221Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network modelPaul eChorley0Anil K Seth1The University of SussexThe University of SussexDopaminergic neurons in the mammalian substantia nigra displaycharacteristic phasic responses to stimuli which reliably predict thereceipt of primary rewards. These responses have been suggested toencode reward prediction-errors similar to those used in reinforcementlearning. Here, we propose a model of dopaminergic activity in whichprediction error signals are generated by the joint action ofshort-latency excitation and long-latency inhibition, in a networkundergoing dopaminergic neuromodulation of both spike-timing dependentsynaptic plasticity and neuronal excitability. In contrast toprevious models, sensitivity to recent events is maintained by theselective modification of specific striatal synapses, efferent tocortical neurons exhibiting stimulus-specific, temporally extendedactivity patterns. Our model shows, in the presence of significantbackground activity, (i) a shift in dopaminergic response from rewardto reward predicting stimuli, (ii) preservation of a response tounexpected rewards, and (iii) a precisely-timed below-baseline dip inactivity observed when expected rewards are omitted.http://journal.frontiersin.org/Journal/10.3389/fncom.2011.00021/fullBasal GangliaDopaminePrefrontal Cortexreinforcement learningSTDPNeuronal excitability |
| spellingShingle | Paul eChorley Anil K Seth Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model Frontiers in Computational Neuroscience Basal Ganglia Dopamine Prefrontal Cortex reinforcement learning STDP Neuronal excitability |
| title | Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model |
| title_full | Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model |
| title_fullStr | Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model |
| title_full_unstemmed | Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model |
| title_short | Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model |
| title_sort | dopamine signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model |
| topic | Basal Ganglia Dopamine Prefrontal Cortex reinforcement learning STDP Neuronal excitability |
| url | http://journal.frontiersin.org/Journal/10.3389/fncom.2011.00021/full |
| work_keys_str_mv | AT paulechorley dopaminesignalledrewardpredictionsgeneratedbycompetitiveexcitationandinhibitioninaspikingneuralnetworkmodel AT anilkseth dopaminesignalledrewardpredictionsgeneratedbycompetitiveexcitationandinhibitioninaspikingneuralnetworkmodel |