Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach
Unmanned surface vehicles (USVs) are intelligent platforms for unmanned surface navigation based on artificial intelligence, motion control, environmental awareness, and other professional technologies. Obstacle avoidance is an important part of its autonomous navigation. Although the USV works in t...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2024-04-01
|
| Series: | Measurement + Control |
| Online Access: | https://doi.org/10.1177/00202940231195937 |
| _version_ | 1797262316029345792 |
|---|---|
| author | Dengyao Jiang Mingzhe Yuan Junfeng Xiong Jinchao Xiao Yong Duan |
| author_facet | Dengyao Jiang Mingzhe Yuan Junfeng Xiong Jinchao Xiao Yong Duan |
| author_sort | Dengyao Jiang |
| collection | DOAJ |
| description | Unmanned surface vehicles (USVs) are intelligent platforms for unmanned surface navigation based on artificial intelligence, motion control, environmental awareness, and other professional technologies. Obstacle avoidance is an important part of its autonomous navigation. Although the USV works in the water environment (e.g. monitoring and tracking, search and rescue scenarios), the dynamic and complex operating environment makes the traditional methods not suitable for solving the obstacle avoidance problem of the USV. In this paper, to address the issue of poor convergence of the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm of Deep Reinforcement Learning (DRL) in an unstructured environment and wave current interference, random walk policy is proposed to deposit the pre-exploration policy of the algorithm into the experience pool to accelerate the convergence of the algorithm and thus achieve USV obstacle avoidance, which can achieve collision-free navigation from any start point to a given end point in a dynamic and complex environment without offline trajectory and track point generation. We design a pre-exploration policy for the environment and a virtual simulation environment for training and testing the algorithm and give the reward function and training method. The simulation results show that our proposed algorithm is more manageable to converge than the original algorithm and can perform better in complex environments in terms of obstacle avoidance behavior, reflecting the algorithm’s feasibility and effectiveness. |
| first_indexed | 2024-04-24T23:55:10Z |
| format | Article |
| id | doaj.art-80049420927642ddbc8a55d5b7c44032 |
| institution | Directory Open Access Journal |
| issn | 0020-2940 |
| language | English |
| last_indexed | 2024-04-24T23:55:10Z |
| publishDate | 2024-04-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Measurement + Control |
| spelling | doaj.art-80049420927642ddbc8a55d5b7c440322024-03-14T14:03:34ZengSAGE PublishingMeasurement + Control0020-29402024-04-015710.1177/00202940231195937Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approachDengyao Jiang0Mingzhe Yuan1Junfeng Xiong2Jinchao Xiao3Yong Duan4Guangzhou Institute of Industrial Intelligence, Guangzhou, ChinaAcademicians Experts Workstation, Guangzhou Institute of Industrial Intelligence, Guangzhou, ChinaAcademicians Experts Workstation, Guangzhou Institute of Industrial Intelligence, Guangzhou, ChinaAcademicians Experts Workstation, Guangzhou Institute of Industrial Intelligence, Guangzhou, ChinaSchool of Information Science and Engineering, Shenyang University of Technology, Shenyang, ChinaUnmanned surface vehicles (USVs) are intelligent platforms for unmanned surface navigation based on artificial intelligence, motion control, environmental awareness, and other professional technologies. Obstacle avoidance is an important part of its autonomous navigation. Although the USV works in the water environment (e.g. monitoring and tracking, search and rescue scenarios), the dynamic and complex operating environment makes the traditional methods not suitable for solving the obstacle avoidance problem of the USV. In this paper, to address the issue of poor convergence of the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm of Deep Reinforcement Learning (DRL) in an unstructured environment and wave current interference, random walk policy is proposed to deposit the pre-exploration policy of the algorithm into the experience pool to accelerate the convergence of the algorithm and thus achieve USV obstacle avoidance, which can achieve collision-free navigation from any start point to a given end point in a dynamic and complex environment without offline trajectory and track point generation. We design a pre-exploration policy for the environment and a virtual simulation environment for training and testing the algorithm and give the reward function and training method. The simulation results show that our proposed algorithm is more manageable to converge than the original algorithm and can perform better in complex environments in terms of obstacle avoidance behavior, reflecting the algorithm’s feasibility and effectiveness.https://doi.org/10.1177/00202940231195937 |
| spellingShingle | Dengyao Jiang Mingzhe Yuan Junfeng Xiong Jinchao Xiao Yong Duan Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach Measurement + Control |
| title | Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach |
| title_full | Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach |
| title_fullStr | Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach |
| title_full_unstemmed | Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach |
| title_short | Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach |
| title_sort | obstacle avoidance usv in multi static obstacle environments based on a deep reinforcement learning approach |
| url | https://doi.org/10.1177/00202940231195937 |
| work_keys_str_mv | AT dengyaojiang obstacleavoidanceusvinmultistaticobstacleenvironmentsbasedonadeepreinforcementlearningapproach AT mingzheyuan obstacleavoidanceusvinmultistaticobstacleenvironmentsbasedonadeepreinforcementlearningapproach AT junfengxiong obstacleavoidanceusvinmultistaticobstacleenvironmentsbasedonadeepreinforcementlearningapproach AT jinchaoxiao obstacleavoidanceusvinmultistaticobstacleenvironmentsbasedonadeepreinforcementlearningapproach AT yongduan obstacleavoidanceusvinmultistaticobstacleenvironmentsbasedonadeepreinforcementlearningapproach |