Forgetting Enhances Episodic Control With Structured Memories
Forgetting is a normal process in healthy brains, and evidence suggests that the mammalian brain forgets more than is required based on limitations of mnemonic capacity. Episodic memories, in particular, are liable to be forgotten over time. Researchers have hypothesized that it may be beneficial fo...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-03-01
|
| Series: | Frontiers in Computational Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fncom.2022.757244/full |
| _version_ | 1811332005030264832 |
|---|---|
| author | Annik Yalnizyan-Carson Annik Yalnizyan-Carson Annik Yalnizyan-Carson Blake A. Richards Blake A. Richards Blake A. Richards Blake A. Richards Blake A. Richards |
| author_facet | Annik Yalnizyan-Carson Annik Yalnizyan-Carson Annik Yalnizyan-Carson Blake A. Richards Blake A. Richards Blake A. Richards Blake A. Richards Blake A. Richards |
| author_sort | Annik Yalnizyan-Carson |
| collection | DOAJ |
| description | Forgetting is a normal process in healthy brains, and evidence suggests that the mammalian brain forgets more than is required based on limitations of mnemonic capacity. Episodic memories, in particular, are liable to be forgotten over time. Researchers have hypothesized that it may be beneficial for decision making to forget episodic memories over time. Reinforcement learning offers a normative framework in which to test such hypotheses. Here, we show that a reinforcement learning agent that uses an episodic memory cache to find rewards in maze environments can forget a large percentage of older memories without any performance impairments, if they utilize mnemonic representations that contain structural information about space. Moreover, we show that some forgetting can actually provide a benefit in performance compared to agents with unbounded memories. Our analyses of the agents show that forgetting reduces the influence of outdated information and states which are not frequently visited on the policies produced by the episodic control system. These results support the hypothesis that some degree of forgetting can be beneficial for decision making, which can help to explain why the brain forgets more than is required by capacity limitations. |
| first_indexed | 2024-04-13T16:29:27Z |
| format | Article |
| id | doaj.art-1c2d4efe82234b939b9505a1e1cfe2d9 |
| institution | Directory Open Access Journal |
| issn | 1662-5188 |
| language | English |
| last_indexed | 2024-04-13T16:29:27Z |
| publishDate | 2022-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computational Neuroscience |
| spelling | doaj.art-1c2d4efe82234b939b9505a1e1cfe2d92022-12-22T02:39:37ZengFrontiers Media S.A.Frontiers in Computational Neuroscience1662-51882022-03-011610.3389/fncom.2022.757244757244Forgetting Enhances Episodic Control With Structured MemoriesAnnik Yalnizyan-Carson0Annik Yalnizyan-Carson1Annik Yalnizyan-Carson2Blake A. Richards3Blake A. Richards4Blake A. Richards5Blake A. Richards6Blake A. Richards7Department of Biological Sciences, University of Toronto Scarborough, Toronto, ON, CanadaDepartment of Cell and Systems Biology, University of Toronto, Toronto, ON, CanadaMontreal Institute for Learning Algorithms (MILA), Montreal, QC, CanadaDepartment of Cell and Systems Biology, University of Toronto, Toronto, ON, CanadaMontreal Institute for Learning Algorithms (MILA), Montreal, QC, CanadaMontreal Neurological Institute, Montreal, QC, CanadaDepartment of Neurology and Neurosurgery, McGill University, Montreal, QC, CanadaSchool of Computer Science, McGill University, Montreal, QC, CanadaForgetting is a normal process in healthy brains, and evidence suggests that the mammalian brain forgets more than is required based on limitations of mnemonic capacity. Episodic memories, in particular, are liable to be forgotten over time. Researchers have hypothesized that it may be beneficial for decision making to forget episodic memories over time. Reinforcement learning offers a normative framework in which to test such hypotheses. Here, we show that a reinforcement learning agent that uses an episodic memory cache to find rewards in maze environments can forget a large percentage of older memories without any performance impairments, if they utilize mnemonic representations that contain structural information about space. Moreover, we show that some forgetting can actually provide a benefit in performance compared to agents with unbounded memories. Our analyses of the agents show that forgetting reduces the influence of outdated information and states which are not frequently visited on the policies produced by the episodic control system. These results support the hypothesis that some degree of forgetting can be beneficial for decision making, which can help to explain why the brain forgets more than is required by capacity limitations.https://www.frontiersin.org/articles/10.3389/fncom.2022.757244/fullreinforcement learningepisodic memorynavigationforgettingsuccessor representations |
| spellingShingle | Annik Yalnizyan-Carson Annik Yalnizyan-Carson Annik Yalnizyan-Carson Blake A. Richards Blake A. Richards Blake A. Richards Blake A. Richards Blake A. Richards Forgetting Enhances Episodic Control With Structured Memories Frontiers in Computational Neuroscience reinforcement learning episodic memory navigation forgetting successor representations |
| title | Forgetting Enhances Episodic Control With Structured Memories |
| title_full | Forgetting Enhances Episodic Control With Structured Memories |
| title_fullStr | Forgetting Enhances Episodic Control With Structured Memories |
| title_full_unstemmed | Forgetting Enhances Episodic Control With Structured Memories |
| title_short | Forgetting Enhances Episodic Control With Structured Memories |
| title_sort | forgetting enhances episodic control with structured memories |
| topic | reinforcement learning episodic memory navigation forgetting successor representations |
| url | https://www.frontiersin.org/articles/10.3389/fncom.2022.757244/full |
| work_keys_str_mv | AT annikyalnizyancarson forgettingenhancesepisodiccontrolwithstructuredmemories AT annikyalnizyancarson forgettingenhancesepisodiccontrolwithstructuredmemories AT annikyalnizyancarson forgettingenhancesepisodiccontrolwithstructuredmemories AT blakearichards forgettingenhancesepisodiccontrolwithstructuredmemories AT blakearichards forgettingenhancesepisodiccontrolwithstructuredmemories AT blakearichards forgettingenhancesepisodiccontrolwithstructuredmemories AT blakearichards forgettingenhancesepisodiccontrolwithstructuredmemories AT blakearichards forgettingenhancesepisodiccontrolwithstructuredmemories |