Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control
Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and ex...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2014-10-01
|
| Series: | Frontiers in Neural Circuits |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fncir.2014.00126/full |
| _version_ | 1811314011825766400 |
|---|---|
| author | Sakyasingha eDasgupta Florentin eWörgötter Poramate eManoonpong Poramate eManoonpong |
| author_facet | Sakyasingha eDasgupta Florentin eWörgötter Poramate eManoonpong Poramate eManoonpong |
| author_sort | Sakyasingha eDasgupta |
| collection | DOAJ |
| description | Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point towards their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms. |
| first_indexed | 2024-04-13T11:04:30Z |
| format | Article |
| id | doaj.art-73b617fca9a84a708118dac161bf5428 |
| institution | Directory Open Access Journal |
| issn | 1662-5110 |
| language | English |
| last_indexed | 2024-04-13T11:04:30Z |
| publishDate | 2014-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neural Circuits |
| spelling | doaj.art-73b617fca9a84a708118dac161bf54282022-12-22T02:49:19ZengFrontiers Media S.A.Frontiers in Neural Circuits1662-51102014-10-01810.3389/fncir.2014.00126107187Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior ControlSakyasingha eDasgupta0Florentin eWörgötter1Poramate eManoonpong2Poramate eManoonpong3Bernstein Centre for Computational Neuroscience, George-August UniversityBernstein Centre for Computational Neuroscience, George-August UniversityBernstein Centre for Computational Neuroscience, George-August UniversityMaersk Mc-Kinney Moller Institute, University of Southern DenmarkGoal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point towards their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.http://journal.frontiersin.org/Journal/10.3389/fncir.2014.00126/fullBasal GangliaCerebellumDecision MakingNeuromodulationNeural controloperant conditioning |
| spellingShingle | Sakyasingha eDasgupta Florentin eWörgötter Poramate eManoonpong Poramate eManoonpong Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control Frontiers in Neural Circuits Basal Ganglia Cerebellum Decision Making Neuromodulation Neural control operant conditioning |
| title | Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control |
| title_full | Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control |
| title_fullStr | Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control |
| title_full_unstemmed | Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control |
| title_short | Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control |
| title_sort | neuromodulatory adaptive combination of correlation based learning in cerebellum and reward based learning in basal ganglia for goal directed behavior control |
| topic | Basal Ganglia Cerebellum Decision Making Neuromodulation Neural control operant conditioning |
| url | http://journal.frontiersin.org/Journal/10.3389/fncir.2014.00126/full |
| work_keys_str_mv | AT sakyasinghaedasgupta neuromodulatoryadaptivecombinationofcorrelationbasedlearningincerebellumandrewardbasedlearninginbasalgangliaforgoaldirectedbehaviorcontrol AT florentineworgotter neuromodulatoryadaptivecombinationofcorrelationbasedlearningincerebellumandrewardbasedlearninginbasalgangliaforgoaldirectedbehaviorcontrol AT poramateemanoonpong neuromodulatoryadaptivecombinationofcorrelationbasedlearningincerebellumandrewardbasedlearninginbasalgangliaforgoaldirectedbehaviorcontrol AT poramateemanoonpong neuromodulatoryadaptivecombinationofcorrelationbasedlearningincerebellumandrewardbasedlearninginbasalgangliaforgoaldirectedbehaviorcontrol |