Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients
This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We offer an instrumental view of action selection, in which an ac...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2019-01-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnbot.2018.00088/full |
| _version_ | 1818386502139248640 |
|---|---|
| author | Yen Yu Acer Y. C. Chang Ryota Kanai |
| author_facet | Yen Yu Acer Y. C. Chang Ryota Kanai |
| author_sort | Yen Yu |
| collection | DOAJ |
| description | This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We offer an instrumental view of action selection, in which an action serves to disclose outcomes that have intrinsic meaningfulness to an agent itself. This motivated two central algorithmic ingredients: devaluation and devaluation progress, both underpin agent's cognition concerning intrinsically generated rewards. The two serve as an instantiation of homeostatic and heterostatic intrinsic motivation. A key insight from our algorithm is that the two seemingly opposite motivations can be reconciled—without which exploration and information-gathering cannot be effectively carried out. We supported this claim with empirical evidence, showing that boredom-enabled agents consistently outperformed other curious or explorative agent variants in model building benchmarks based on self-assisted experience accumulation. |
| first_indexed | 2024-12-14T03:55:04Z |
| format | Article |
| id | doaj.art-5ab31f5070b64be986ee3787c6ecac87 |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-12-14T03:55:04Z |
| publishDate | 2019-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-5ab31f5070b64be986ee3787c6ecac872022-12-21T23:18:07ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182019-01-011210.3389/fnbot.2018.00088391132Boredom-Driven Curious Learning by Homeo-Heterostatic Value GradientsYen YuAcer Y. C. ChangRyota KanaiThis paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We offer an instrumental view of action selection, in which an action serves to disclose outcomes that have intrinsic meaningfulness to an agent itself. This motivated two central algorithmic ingredients: devaluation and devaluation progress, both underpin agent's cognition concerning intrinsically generated rewards. The two serve as an instantiation of homeostatic and heterostatic intrinsic motivation. A key insight from our algorithm is that the two seemingly opposite motivations can be reconciled—without which exploration and information-gathering cannot be effectively carried out. We supported this claim with empirical evidence, showing that boredom-enabled agents consistently outperformed other curious or explorative agent variants in model building benchmarks based on self-assisted experience accumulation.https://www.frontiersin.org/article/10.3389/fnbot.2018.00088/fullcuriosityboredomgoal-directednessintrinsic motivationoutcome devaluationsatiety |
| spellingShingle | Yen Yu Acer Y. C. Chang Ryota Kanai Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients Frontiers in Neurorobotics curiosity boredom goal-directedness intrinsic motivation outcome devaluation satiety |
| title | Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients |
| title_full | Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients |
| title_fullStr | Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients |
| title_full_unstemmed | Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients |
| title_short | Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients |
| title_sort | boredom driven curious learning by homeo heterostatic value gradients |
| topic | curiosity boredom goal-directedness intrinsic motivation outcome devaluation satiety |
| url | https://www.frontiersin.org/article/10.3389/fnbot.2018.00088/full |
| work_keys_str_mv | AT yenyu boredomdrivencuriouslearningbyhomeoheterostaticvaluegradients AT acerycchang boredomdrivencuriouslearningbyhomeoheterostaticvaluegradients AT ryotakanai boredomdrivencuriouslearningbyhomeoheterostaticvaluegradients |