Visual navigation with multiple goals based on deep reinforcement learning
Learning to adapt to a series of different goals in visual navigation is challenging. In this work, we present a model-embedded actor-critic architecture for the multigoal visual navigation task. To enhance the task cooperation in multigoal learning, we introduce two new designs to the reinforcement...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/163231 |
| _version_ | 1826130242615377920 |
|---|---|
| author | Rao, Zhenhuan Wu, Yuechen Yang, Zifei Zhang, Wei Lu, Shijian Lu, Weizhi Zha, ZhengJun |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Rao, Zhenhuan Wu, Yuechen Yang, Zifei Zhang, Wei Lu, Shijian Lu, Weizhi Zha, ZhengJun |
| author_sort | Rao, Zhenhuan |
| collection | NTU |
| description | Learning to adapt to a series of different goals in visual navigation is challenging. In this work, we present a model-embedded actor-critic architecture for the multigoal visual navigation task. To enhance the task cooperation in multigoal learning, we introduce two new designs to the reinforcement learning scheme: inverse dynamics model (InvDM) and multigoal colearning (MgCl). Specifically, InvDM is proposed to capture the navigation-relevant association between state and goal and provide additional training signals to relieve the sparse reward issue. MgCl aims at improving the sample efficiency and supports the agent to learn from unintentional positive experiences. Besides, to further improve the scene generalization capability of the agent, we present an enhanced navigation model that consists of two self-supervised auxiliary task modules. The first module, which is named path closed-loop detection, helps to understand whether the state has been experienced. The second one, namely the state-target matching module, tries to figure out the difference between state and goal. Extensive results on the interactive platform AI2-THOR demonstrate that the agent trained with the proposed method converges faster than state-of-the-art methods while owning good generalization capability. The video demonstration is available at https://vsislab.github.io/mgvn. |
| first_indexed | 2024-10-01T07:53:16Z |
| format | Journal Article |
| id | ntu-10356/163231 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:53:16Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1632312022-11-29T05:03:31Z Visual navigation with multiple goals based on deep reinforcement learning Rao, Zhenhuan Wu, Yuechen Yang, Zifei Zhang, Wei Lu, Shijian Lu, Weizhi Zha, ZhengJun School of Computer Science and Engineering Engineering::Computer science and engineering Deep Reinforcement Learning Scene Generalization Learning to adapt to a series of different goals in visual navigation is challenging. In this work, we present a model-embedded actor-critic architecture for the multigoal visual navigation task. To enhance the task cooperation in multigoal learning, we introduce two new designs to the reinforcement learning scheme: inverse dynamics model (InvDM) and multigoal colearning (MgCl). Specifically, InvDM is proposed to capture the navigation-relevant association between state and goal and provide additional training signals to relieve the sparse reward issue. MgCl aims at improving the sample efficiency and supports the agent to learn from unintentional positive experiences. Besides, to further improve the scene generalization capability of the agent, we present an enhanced navigation model that consists of two self-supervised auxiliary task modules. The first module, which is named path closed-loop detection, helps to understand whether the state has been experienced. The second one, namely the state-target matching module, tries to figure out the difference between state and goal. Extensive results on the interactive platform AI2-THOR demonstrate that the agent trained with the proposed method converges faster than state-of-the-art methods while owning good generalization capability. The video demonstration is available at https://vsislab.github.io/mgvn. This work was supported in part by the National Key Research and Development Plan of China under Grant 2018AAA0102504; in part by the National Natural Science Foundation of China under Grant U1913204, Grant 61991411, and Grant U19B2038; in part by the Natural Science Foundation of Shandong Province for Distinguished Young Scholars under Grant ZR2020JQ29; and in part by the Shandong Major Scientific and Technological Innovation Project (MSTIP) under Grant 2018CXGC1503. 2022-11-29T05:03:31Z 2022-11-29T05:03:31Z 2021 Journal Article Rao, Z., Wu, Y., Yang, Z., Zhang, W., Lu, S., Lu, W. & Zha, Z. (2021). Visual navigation with multiple goals based on deep reinforcement learning. IEEE Transactions On Neural Networks and Learning Systems, 32(12), 5445-5455. https://dx.doi.org/10.1109/TNNLS.2021.3057424 2162-237X https://hdl.handle.net/10356/163231 10.1109/TNNLS.2021.3057424 33667168 2-s2.0-85102271666 12 32 5445 5455 en IEEE Transactions on Neural Networks and Learning Systems © 2021 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Deep Reinforcement Learning Scene Generalization Rao, Zhenhuan Wu, Yuechen Yang, Zifei Zhang, Wei Lu, Shijian Lu, Weizhi Zha, ZhengJun Visual navigation with multiple goals based on deep reinforcement learning |
| title | Visual navigation with multiple goals based on deep reinforcement learning |
| title_full | Visual navigation with multiple goals based on deep reinforcement learning |
| title_fullStr | Visual navigation with multiple goals based on deep reinforcement learning |
| title_full_unstemmed | Visual navigation with multiple goals based on deep reinforcement learning |
| title_short | Visual navigation with multiple goals based on deep reinforcement learning |
| title_sort | visual navigation with multiple goals based on deep reinforcement learning |
| topic | Engineering::Computer science and engineering Deep Reinforcement Learning Scene Generalization |
| url | https://hdl.handle.net/10356/163231 |
| work_keys_str_mv | AT raozhenhuan visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning AT wuyuechen visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning AT yangzifei visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning AT zhangwei visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning AT lushijian visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning AT luweizhi visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning AT zhazhengjun visualnavigationwithmultiplegoalsbasedondeepreinforcementlearning |