BND*-DDQN: learn to steer autonomously through deep reinforcement learning
It is vital for mobile robots to achieve safe autonomous steering in various changing environments. In this paper, a novel end-to-end network architecture is proposed for mobile robots to learn steering autonomously through deep reinforcement learning. Specifically, two sets of feature representatio...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/159818 |
| _version_ | 1811684470063890432 |
|---|---|
| author | Wu, Keyu Wang, Han Esfahani, Mahdi Abolfazli Yuan, Shenghai |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Wu, Keyu Wang, Han Esfahani, Mahdi Abolfazli Yuan, Shenghai |
| author_sort | Wu, Keyu |
| collection | NTU |
| description | It is vital for mobile robots to achieve safe autonomous steering in various changing environments. In this paper, a novel end-to-end network architecture is proposed for mobile robots to learn steering autonomously through deep reinforcement learning. Specifically, two sets of feature representations are first extracted from the depth inputs through two different input streams. The acquired features are then merged together to derive both linear and angular actions simultaneously. Moreover, a new action selection strategy is also introduced to achieve motion filtering by taking the consistency in angular velocity into account. Besides, in addition to the extrinsic rewards, the intrinsic bonuses are also adopted during training to improve the exploration capability. Furthermore, it is worth noting the proposed model is readily transferable from the simple virtual training environment to much more complicated real-world scenarios so that no further fine-tuning is required for real deployment. Compared to the existing methods, the proposed method demonstrates significant superiority in terms of average reward, convergence speed, success rate, and generalization capability. In addition, it exhibits outstanding performance in various cluttered real-world environments containing both static and dynamic obstacles. A video of our experiments can be found at https://youtu.be/19jrQGG1oCU. |
| first_indexed | 2024-10-01T04:29:08Z |
| format | Journal Article |
| id | ntu-10356/159818 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:29:08Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1598182022-07-04T03:25:00Z BND*-DDQN: learn to steer autonomously through deep reinforcement learning Wu, Keyu Wang, Han Esfahani, Mahdi Abolfazli Yuan, Shenghai School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Feature Extraction Training It is vital for mobile robots to achieve safe autonomous steering in various changing environments. In this paper, a novel end-to-end network architecture is proposed for mobile robots to learn steering autonomously through deep reinforcement learning. Specifically, two sets of feature representations are first extracted from the depth inputs through two different input streams. The acquired features are then merged together to derive both linear and angular actions simultaneously. Moreover, a new action selection strategy is also introduced to achieve motion filtering by taking the consistency in angular velocity into account. Besides, in addition to the extrinsic rewards, the intrinsic bonuses are also adopted during training to improve the exploration capability. Furthermore, it is worth noting the proposed model is readily transferable from the simple virtual training environment to much more complicated real-world scenarios so that no further fine-tuning is required for real deployment. Compared to the existing methods, the proposed method demonstrates significant superiority in terms of average reward, convergence speed, success rate, and generalization capability. In addition, it exhibits outstanding performance in various cluttered real-world environments containing both static and dynamic obstacles. A video of our experiments can be found at https://youtu.be/19jrQGG1oCU. Nanyang Technological University National Research Foundation (NRF) This work was supported in part by ST Engineering-NTU Corporate Laboratory and in part by NRF. 2022-07-04T03:25:00Z 2022-07-04T03:25:00Z 2019 Journal Article Wu, K., Wang, H., Esfahani, M. A. & Yuan, S. (2019). BND*-DDQN: learn to steer autonomously through deep reinforcement learning. IEEE Transactions On Cognitive and Developmental Systems, 13(2), 249-261. https://dx.doi.org/10.1109/TCDS.2019.2928820 2379-8920 https://hdl.handle.net/10356/159818 10.1109/TCDS.2019.2928820 2-s2.0-85069924995 2 13 249 261 en IEEE Transactions on Cognitive and Developmental Systems © 2019 IEEE. All rights reserved. |
| spellingShingle | Engineering::Electrical and electronic engineering Feature Extraction Training Wu, Keyu Wang, Han Esfahani, Mahdi Abolfazli Yuan, Shenghai BND*-DDQN: learn to steer autonomously through deep reinforcement learning |
| title | BND*-DDQN: learn to steer autonomously through deep reinforcement learning |
| title_full | BND*-DDQN: learn to steer autonomously through deep reinforcement learning |
| title_fullStr | BND*-DDQN: learn to steer autonomously through deep reinforcement learning |
| title_full_unstemmed | BND*-DDQN: learn to steer autonomously through deep reinforcement learning |
| title_short | BND*-DDQN: learn to steer autonomously through deep reinforcement learning |
| title_sort | bnd ddqn learn to steer autonomously through deep reinforcement learning |
| topic | Engineering::Electrical and electronic engineering Feature Extraction Training |
| url | https://hdl.handle.net/10356/159818 |
| work_keys_str_mv | AT wukeyu bndddqnlearntosteerautonomouslythroughdeepreinforcementlearning AT wanghan bndddqnlearntosteerautonomouslythroughdeepreinforcementlearning AT esfahanimahdiabolfazli bndddqnlearntosteerautonomouslythroughdeepreinforcementlearning AT yuanshenghai bndddqnlearntosteerautonomouslythroughdeepreinforcementlearning |