A neural model of hierarchical reinforcement learning.
We develop a novel, biologically detailed neural model of reinforcement learning (RL) processes in the brain. This model incorporates a broad range of biological features that pose challenges to neural RL, such as temporally extended action sequences, continuous environments involving unknown time d...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2017-01-01
|
| Series: | PLoS ONE |
| Online Access: | http://europepmc.org/articles/PMC5500327?pdf=render |
| _version_ | 1818126528473464832 |
|---|---|
| author | Daniel Rasmussen Aaron Voelker Chris Eliasmith |
| author_facet | Daniel Rasmussen Aaron Voelker Chris Eliasmith |
| author_sort | Daniel Rasmussen |
| collection | DOAJ |
| description | We develop a novel, biologically detailed neural model of reinforcement learning (RL) processes in the brain. This model incorporates a broad range of biological features that pose challenges to neural RL, such as temporally extended action sequences, continuous environments involving unknown time delays, and noisy/imprecise computations. Most significantly, we expand the model into the realm of hierarchical reinforcement learning (HRL), which divides the RL process into a hierarchy of actions at different levels of abstraction. Here we implement all the major components of HRL in a neural model that captures a variety of known anatomical and physiological properties of the brain. We demonstrate the performance of the model in a range of different environments, in order to emphasize the aim of understanding the brain's general reinforcement learning ability. These results show that the model compares well to previous modelling work and demonstrates improved performance as a result of its hierarchical ability. We also show that the model's behaviour is consistent with available data on human hierarchical RL, and generate several novel predictions. |
| first_indexed | 2024-12-11T07:02:54Z |
| format | Article |
| id | doaj.art-a60fd229ec3b4e7e9b643d4b4ba2bc14 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-11T07:02:54Z |
| publishDate | 2017-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-a60fd229ec3b4e7e9b643d4b4ba2bc142022-12-22T01:16:34ZengPublic Library of Science (PLoS)PLoS ONE1932-62032017-01-01127e018023410.1371/journal.pone.0180234A neural model of hierarchical reinforcement learning.Daniel RasmussenAaron VoelkerChris EliasmithWe develop a novel, biologically detailed neural model of reinforcement learning (RL) processes in the brain. This model incorporates a broad range of biological features that pose challenges to neural RL, such as temporally extended action sequences, continuous environments involving unknown time delays, and noisy/imprecise computations. Most significantly, we expand the model into the realm of hierarchical reinforcement learning (HRL), which divides the RL process into a hierarchy of actions at different levels of abstraction. Here we implement all the major components of HRL in a neural model that captures a variety of known anatomical and physiological properties of the brain. We demonstrate the performance of the model in a range of different environments, in order to emphasize the aim of understanding the brain's general reinforcement learning ability. These results show that the model compares well to previous modelling work and demonstrates improved performance as a result of its hierarchical ability. We also show that the model's behaviour is consistent with available data on human hierarchical RL, and generate several novel predictions.http://europepmc.org/articles/PMC5500327?pdf=render |
| spellingShingle | Daniel Rasmussen Aaron Voelker Chris Eliasmith A neural model of hierarchical reinforcement learning. PLoS ONE |
| title | A neural model of hierarchical reinforcement learning. |
| title_full | A neural model of hierarchical reinforcement learning. |
| title_fullStr | A neural model of hierarchical reinforcement learning. |
| title_full_unstemmed | A neural model of hierarchical reinforcement learning. |
| title_short | A neural model of hierarchical reinforcement learning. |
| title_sort | neural model of hierarchical reinforcement learning |
| url | http://europepmc.org/articles/PMC5500327?pdf=render |
| work_keys_str_mv | AT danielrasmussen aneuralmodelofhierarchicalreinforcementlearning AT aaronvoelker aneuralmodelofhierarchicalreinforcementlearning AT chriseliasmith aneuralmodelofhierarchicalreinforcementlearning AT danielrasmussen neuralmodelofhierarchicalreinforcementlearning AT aaronvoelker neuralmodelofhierarchicalreinforcementlearning AT chriseliasmith neuralmodelofhierarchicalreinforcementlearning |