Efficient Novelty Search Through Deep Reinforcement Learning
Novelty search, which was inspired by the nature that evolves creatures with diversity, has shown great potential in solving reinforcement learning (RL) tasks with sparse and deceptive rewards. However, most of the existing novelty search methods evolve the populations through hybrization and mutati...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9139203/ |
| _version_ | 1818853519935930368 |
|---|---|
| author | Longxiang Shi Shijian Li Qian Zheng Min Yao Gang Pan |
| author_facet | Longxiang Shi Shijian Li Qian Zheng Min Yao Gang Pan |
| author_sort | Longxiang Shi |
| collection | DOAJ |
| description | Novelty search, which was inspired by the nature that evolves creatures with diversity, has shown great potential in solving reinforcement learning (RL) tasks with sparse and deceptive rewards. However, most of the existing novelty search methods evolve the populations through hybrization and mutation, which is inefficient in diverging populations. In this paper, we propose a method which incorporates deep RL with novelty search to improve the efficiency of diverging the populations for novelty search. We first propose a strategy that improves the novelty of individuals generated by genetic algorithm using reinforcement learning. Based on this strategy, we propose a framework that incorporates deep RL with novelty search, and then derive an algorithm to improve the search efficiency of the novelty search for continuous control tasks. Our experimental results show that our method can improve the search efficiency of novelty search and can also provide a competitive performance compared to some of the existing novelty search methods. The implementation of our method is available at: https://github.com/shilx001/NoveltySearch_Improvement. |
| first_indexed | 2024-12-19T07:38:07Z |
| format | Article |
| id | doaj.art-43d962540f3c4eea9a3757dec2e3355f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T07:38:07Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-43d962540f3c4eea9a3757dec2e3355f2022-12-21T20:30:32ZengIEEEIEEE Access2169-35362020-01-01812880912881810.1109/ACCESS.2020.30087359139203Efficient Novelty Search Through Deep Reinforcement LearningLongxiang Shi0https://orcid.org/0000-0003-4334-1182Shijian Li1Qian Zheng2Min Yao3Gang Pan4https://orcid.org/0000-0002-4049-6181College of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaSchool of Electrical and Electronic Engineering, Nanyang Technological University, SingaporeCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaNovelty search, which was inspired by the nature that evolves creatures with diversity, has shown great potential in solving reinforcement learning (RL) tasks with sparse and deceptive rewards. However, most of the existing novelty search methods evolve the populations through hybrization and mutation, which is inefficient in diverging populations. In this paper, we propose a method which incorporates deep RL with novelty search to improve the efficiency of diverging the populations for novelty search. We first propose a strategy that improves the novelty of individuals generated by genetic algorithm using reinforcement learning. Based on this strategy, we propose a framework that incorporates deep RL with novelty search, and then derive an algorithm to improve the search efficiency of the novelty search for continuous control tasks. Our experimental results show that our method can improve the search efficiency of novelty search and can also provide a competitive performance compared to some of the existing novelty search methods. The implementation of our method is available at: https://github.com/shilx001/NoveltySearch_Improvement.https://ieeexplore.ieee.org/document/9139203/Reinforcement learningnovelty searchevolutionary computingdeep learning |
| spellingShingle | Longxiang Shi Shijian Li Qian Zheng Min Yao Gang Pan Efficient Novelty Search Through Deep Reinforcement Learning IEEE Access Reinforcement learning novelty search evolutionary computing deep learning |
| title | Efficient Novelty Search Through Deep Reinforcement Learning |
| title_full | Efficient Novelty Search Through Deep Reinforcement Learning |
| title_fullStr | Efficient Novelty Search Through Deep Reinforcement Learning |
| title_full_unstemmed | Efficient Novelty Search Through Deep Reinforcement Learning |
| title_short | Efficient Novelty Search Through Deep Reinforcement Learning |
| title_sort | efficient novelty search through deep reinforcement learning |
| topic | Reinforcement learning novelty search evolutionary computing deep learning |
| url | https://ieeexplore.ieee.org/document/9139203/ |
| work_keys_str_mv | AT longxiangshi efficientnoveltysearchthroughdeepreinforcementlearning AT shijianli efficientnoveltysearchthroughdeepreinforcementlearning AT qianzheng efficientnoveltysearchthroughdeepreinforcementlearning AT minyao efficientnoveltysearchthroughdeepreinforcementlearning AT gangpan efficientnoveltysearchthroughdeepreinforcementlearning |