Neural network models of hierarchical map-based planning in the brain
<p>Many researchers have tried to model how model-based behaviour is produced in the brain, and specifically how environmental knowledge is learned and used in the form of cognitive maps. Previous work has proposed various cellular substrates that might encode a cognitive map, as well as se...
| Egile nagusia: | |
|---|---|
| Beste egile batzuk: | |
| Formatua: | Thesis |
| Hizkuntza: | English |
| Argitaratua: |
2018
|
| Gaiak: |
| _version_ | 1826298106931576832 |
|---|---|
| author | Jordan, H |
| author2 | Stringer, S |
| author_facet | Stringer, S Jordan, H |
| author_sort | Jordan, H |
| collection | OXFORD |
| description | <p>Many researchers have tried to model how model-based behaviour is produced in the brain, and specifically how environmental knowledge is learned and used in the form of cognitive maps. Previous work has proposed various cellular substrates that might encode a cognitive map, as well as several possible mechanisms for self-organising these maps. Current models also agree to some extent on the planning mechanism that produces goal-directed actions using a cognitive map, although they propose subtly different implementations of this planning mechanism, and a variety of mechanisms for extending it. We felt that this previous work was limited in several important ways: there was little consensus on how these cognitive maps were represented and how they were formed, the planning mechanism that was used seemed to be inherently limited to performing relatively simple tasks, and there had been little consideration of how these mechanisms would scale up beyond laboratory tasks.</p> <p>This thesis makes several important advances. Firstly, the planning mechanism that is used by the majority of current neural models on this subject propagates a decaying signal through the environment to create a gradient that points towards the goal. However, this decaying signal will eventually decay to nothing, limiting the scale and complexity of tasks that can be solved in this manner. Based on the idea, originally proposed by Ponulak et al. 2013, that a wavefront propagating from a goal state can carry information about the direction of that goal state, we propose a novel planning mechanism that does not require decaying activity.</p> <p>Unlike the model proposed by Ponulak et al., the mechanism that we propose is: able to work in a state-action map, able to output explicit actions to move towards the goal, able to perform planning without altering synaptic weights, able to plan in nondeterministic environments, and able to interface with a hierarchical mechanism. We also propose several ways in which a network which implements this planning mechanism can self-organise during unsupervised exploration. We demonstrate that the neural substrate for the cognitive map, the mechanisms required to plan using this map and the cognitive map itself can all be learned using local Hebbian learning, in an unsupervised manner.We compare and contrast these mechanisms in terms of both their effectiveness and their biological plausibility.</p> <p>Finally, we extend the model with a hierarchical planning mechanism: a layer of cells that can identify frequently-used sequences of actions and reuse them to significantly increase the efficiency of planning. This increase in efficiency is largest for tasks with longer, more complex solutions. We speculate that this may explain the apparent ability of humans and animals to perform model-based planning on both small scales (laboratory experiments) and on large scales (outdoor travel) without a noticeable loss of efficiency.</p> |
| first_indexed | 2024-03-07T04:41:47Z |
| format | Thesis |
| id | oxford-uuid:d1e5dcf8-2721-440c-b27f-5b56f33eff21 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T04:41:47Z |
| publishDate | 2018 |
| record_format | dspace |
| spelling | oxford-uuid:d1e5dcf8-2721-440c-b27f-5b56f33eff212022-03-27T08:00:04ZNeural network models of hierarchical map-based planning in the brainThesishttp://purl.org/coar/resource_type/c_db06uuid:d1e5dcf8-2721-440c-b27f-5b56f33eff21Computational neuroscienceEnglishORA Deposit2018Jordan, HStringer, S<p>Many researchers have tried to model how model-based behaviour is produced in the brain, and specifically how environmental knowledge is learned and used in the form of cognitive maps. Previous work has proposed various cellular substrates that might encode a cognitive map, as well as several possible mechanisms for self-organising these maps. Current models also agree to some extent on the planning mechanism that produces goal-directed actions using a cognitive map, although they propose subtly different implementations of this planning mechanism, and a variety of mechanisms for extending it. We felt that this previous work was limited in several important ways: there was little consensus on how these cognitive maps were represented and how they were formed, the planning mechanism that was used seemed to be inherently limited to performing relatively simple tasks, and there had been little consideration of how these mechanisms would scale up beyond laboratory tasks.</p> <p>This thesis makes several important advances. Firstly, the planning mechanism that is used by the majority of current neural models on this subject propagates a decaying signal through the environment to create a gradient that points towards the goal. However, this decaying signal will eventually decay to nothing, limiting the scale and complexity of tasks that can be solved in this manner. Based on the idea, originally proposed by Ponulak et al. 2013, that a wavefront propagating from a goal state can carry information about the direction of that goal state, we propose a novel planning mechanism that does not require decaying activity.</p> <p>Unlike the model proposed by Ponulak et al., the mechanism that we propose is: able to work in a state-action map, able to output explicit actions to move towards the goal, able to perform planning without altering synaptic weights, able to plan in nondeterministic environments, and able to interface with a hierarchical mechanism. We also propose several ways in which a network which implements this planning mechanism can self-organise during unsupervised exploration. We demonstrate that the neural substrate for the cognitive map, the mechanisms required to plan using this map and the cognitive map itself can all be learned using local Hebbian learning, in an unsupervised manner.We compare and contrast these mechanisms in terms of both their effectiveness and their biological plausibility.</p> <p>Finally, we extend the model with a hierarchical planning mechanism: a layer of cells that can identify frequently-used sequences of actions and reuse them to significantly increase the efficiency of planning. This increase in efficiency is largest for tasks with longer, more complex solutions. We speculate that this may explain the apparent ability of humans and animals to perform model-based planning on both small scales (laboratory experiments) and on large scales (outdoor travel) without a noticeable loss of efficiency.</p> |
| spellingShingle | Computational neuroscience Jordan, H Neural network models of hierarchical map-based planning in the brain |
| title | Neural network models of hierarchical map-based planning in the brain |
| title_full | Neural network models of hierarchical map-based planning in the brain |
| title_fullStr | Neural network models of hierarchical map-based planning in the brain |
| title_full_unstemmed | Neural network models of hierarchical map-based planning in the brain |
| title_short | Neural network models of hierarchical map-based planning in the brain |
| title_sort | neural network models of hierarchical map based planning in the brain |
| topic | Computational neuroscience |
| work_keys_str_mv | AT jordanh neuralnetworkmodelsofhierarchicalmapbasedplanninginthebrain |