Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation
Reinforcement learning is a powerful model of animal learning in brief, controlled experimental conditions, but does not readily explain the development of behavior over an animal’s whole lifetime. In this paper, we describe a framework to address this shortcoming by introducing the single-life rein...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Cognitive Science Society
2024
|
| _version_ | 1811139580537077760 |
|---|---|
| author | Sandbrink, K Christian, B Nasvytis, L Schroeder de Witt, C Butlin, P |
| author_facet | Sandbrink, K Christian, B Nasvytis, L Schroeder de Witt, C Butlin, P |
| author_sort | Sandbrink, K |
| collection | OXFORD |
| description | Reinforcement learning is a powerful model of animal learning in brief, controlled experimental conditions, but does not readily explain the development of behavior over an animal’s whole lifetime. In this paper, we describe a framework to address this shortcoming by introducing the single-life reinforcement learning setting to cognitive science. We construct an agent with two learning systems: an extrinsic learner that learns within a single lifetime, and an intrinsic learner that learns across lifetimes, equipping the agent with intrinsic motivation. We show that this model outperforms heuristic benchmarks and recapitulates a transition from exploratory to habit-driven behavior, while allowing the agent to learn an interpretable value function. We formulate a precise definition of intrinsic motivation and discuss the philosophical implications of using reinforcement learning as a model of behavior in the real world |
| first_indexed | 2024-09-25T04:08:21Z |
| format | Conference item |
| id | oxford-uuid:0ca8699b-a631-4b69-8ee0-78b9ba1d47dc |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:08:21Z |
| publishDate | 2024 |
| publisher | Cognitive Science Society |
| record_format | dspace |
| spelling | oxford-uuid:0ca8699b-a631-4b69-8ee0-78b9ba1d47dc2024-06-13T09:41:04ZCan reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivationConference itemhttp://purl.org/coar/resource_type/c_5794uuid:0ca8699b-a631-4b69-8ee0-78b9ba1d47dcEnglishSymplectic ElementsCognitive Science Society2024Sandbrink, KChristian, BNasvytis, LSchroeder de Witt, CButlin, PReinforcement learning is a powerful model of animal learning in brief, controlled experimental conditions, but does not readily explain the development of behavior over an animal’s whole lifetime. In this paper, we describe a framework to address this shortcoming by introducing the single-life reinforcement learning setting to cognitive science. We construct an agent with two learning systems: an extrinsic learner that learns within a single lifetime, and an intrinsic learner that learns across lifetimes, equipping the agent with intrinsic motivation. We show that this model outperforms heuristic benchmarks and recapitulates a transition from exploratory to habit-driven behavior, while allowing the agent to learn an interpretable value function. We formulate a precise definition of intrinsic motivation and discuss the philosophical implications of using reinforcement learning as a model of behavior in the real world |
| spellingShingle | Sandbrink, K Christian, B Nasvytis, L Schroeder de Witt, C Butlin, P Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation |
| title | Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation |
| title_full | Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation |
| title_fullStr | Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation |
| title_full_unstemmed | Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation |
| title_short | Can reinforcement learning model learning across development? Online lifelong learning through adaptive intrinsic motivation |
| title_sort | can reinforcement learning model learning across development online lifelong learning through adaptive intrinsic motivation |
| work_keys_str_mv | AT sandbrinkk canreinforcementlearningmodellearningacrossdevelopmentonlinelifelonglearningthroughadaptiveintrinsicmotivation AT christianb canreinforcementlearningmodellearningacrossdevelopmentonlinelifelonglearningthroughadaptiveintrinsicmotivation AT nasvytisl canreinforcementlearningmodellearningacrossdevelopmentonlinelifelonglearningthroughadaptiveintrinsicmotivation AT schroederdewittc canreinforcementlearningmodellearningacrossdevelopmentonlinelifelonglearningthroughadaptiveintrinsicmotivation AT butlinp canreinforcementlearningmodellearningacrossdevelopmentonlinelifelonglearningthroughadaptiveintrinsicmotivation |