A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons
This research aims to solve the safe navigation problem of autonomous underwater vehicles (AUVs) in deep ocean, which is a complex and changeable environment with various mountains. When an AUV reaches the deep sea navigation, it encounters many underwater canyons, and the hard valley walls threaten...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-02-01
|
| Series: | Journal of Marine Science and Engineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-1312/9/3/252 |
| _version_ | 1797417541114527744 |
|---|---|
| author | Yushan Sun Xiaokun Luo Xiangrui Ran Guocheng Zhang |
| author_facet | Yushan Sun Xiaokun Luo Xiangrui Ran Guocheng Zhang |
| author_sort | Yushan Sun |
| collection | DOAJ |
| description | This research aims to solve the safe navigation problem of autonomous underwater vehicles (AUVs) in deep ocean, which is a complex and changeable environment with various mountains. When an AUV reaches the deep sea navigation, it encounters many underwater canyons, and the hard valley walls threaten its safety seriously. To solve the problem on the safe driving of AUV in underwater canyons and address the potential of AUV autonomous obstacle avoidance in uncertain environments, an improved AUV path planning algorithm based on the deep deterministic policy gradient (DDPG) algorithm is proposed in this work. This method refers to an end-to-end path planning algorithm that optimizes the strategy directly. It takes sensor information as input and driving speed and yaw angle as outputs. The path planning algorithm can reach the predetermined target point while avoiding large-scale static obstacles, such as valley walls in the simulated underwater canyon environment, as well as sudden small-scale dynamic obstacles, such as marine life and other vehicles. In addition, this research aims at the multi-objective structure of the obstacle avoidance of path planning, modularized reward function design, and combined artificial potential field method to set continuous rewards. This research also proposes a new algorithm called deep SumTree-deterministic policy gradient algorithm (SumTree-DDPG), which improves the random storage and extraction strategy of DDPG algorithm experience samples. According to the importance of the experience samples, the samples are classified and stored in combination with the SumTree structure, high-quality samples are extracted continuously, and SumTree-DDPG algorithm finally improves the speed of the convergence model. Finally, this research uses Python language to write an underwater canyon simulation environment and builds a deep reinforcement learning simulation platform on a high-performance computer to conduct simulation learning training for AUV. Data simulation verified that the proposed path planning method can guide the under-actuated underwater robot to navigate to the target without colliding with any obstacles. In comparison with the DDPG algorithm, the stability, training’s total reward, and robustness of the improved Sumtree-DDPG algorithm planner in this study are better. |
| first_indexed | 2024-03-09T06:21:19Z |
| format | Article |
| id | doaj.art-de9a0dd5772c4108a6fe4e46b410a70a |
| institution | Directory Open Access Journal |
| issn | 2077-1312 |
| language | English |
| last_indexed | 2024-03-09T06:21:19Z |
| publishDate | 2021-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Marine Science and Engineering |
| spelling | doaj.art-de9a0dd5772c4108a6fe4e46b410a70a2023-12-03T11:48:37ZengMDPI AGJournal of Marine Science and Engineering2077-13122021-02-019325210.3390/jmse9030252A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater CanyonsYushan Sun0Xiaokun Luo1Xiangrui Ran2Guocheng Zhang3School of Naval Engineering, Harbin Engineering University, Harbin 150001, ChinaSchool of Naval Engineering, Harbin Engineering University, Harbin 150001, ChinaSchool of Naval Engineering, Harbin Engineering University, Harbin 150001, ChinaSchool of Naval Engineering, Harbin Engineering University, Harbin 150001, ChinaThis research aims to solve the safe navigation problem of autonomous underwater vehicles (AUVs) in deep ocean, which is a complex and changeable environment with various mountains. When an AUV reaches the deep sea navigation, it encounters many underwater canyons, and the hard valley walls threaten its safety seriously. To solve the problem on the safe driving of AUV in underwater canyons and address the potential of AUV autonomous obstacle avoidance in uncertain environments, an improved AUV path planning algorithm based on the deep deterministic policy gradient (DDPG) algorithm is proposed in this work. This method refers to an end-to-end path planning algorithm that optimizes the strategy directly. It takes sensor information as input and driving speed and yaw angle as outputs. The path planning algorithm can reach the predetermined target point while avoiding large-scale static obstacles, such as valley walls in the simulated underwater canyon environment, as well as sudden small-scale dynamic obstacles, such as marine life and other vehicles. In addition, this research aims at the multi-objective structure of the obstacle avoidance of path planning, modularized reward function design, and combined artificial potential field method to set continuous rewards. This research also proposes a new algorithm called deep SumTree-deterministic policy gradient algorithm (SumTree-DDPG), which improves the random storage and extraction strategy of DDPG algorithm experience samples. According to the importance of the experience samples, the samples are classified and stored in combination with the SumTree structure, high-quality samples are extracted continuously, and SumTree-DDPG algorithm finally improves the speed of the convergence model. Finally, this research uses Python language to write an underwater canyon simulation environment and builds a deep reinforcement learning simulation platform on a high-performance computer to conduct simulation learning training for AUV. Data simulation verified that the proposed path planning method can guide the under-actuated underwater robot to navigate to the target without colliding with any obstacles. In comparison with the DDPG algorithm, the stability, training’s total reward, and robustness of the improved Sumtree-DDPG algorithm planner in this study are better.https://www.mdpi.com/2077-1312/9/3/252autonomous underwater vehicle2D optimal path planningdeep reinforcement learningunknown underwater canyons environment |
| spellingShingle | Yushan Sun Xiaokun Luo Xiangrui Ran Guocheng Zhang A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons Journal of Marine Science and Engineering autonomous underwater vehicle 2D optimal path planning deep reinforcement learning unknown underwater canyons environment |
| title | A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons |
| title_full | A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons |
| title_fullStr | A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons |
| title_full_unstemmed | A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons |
| title_short | A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons |
| title_sort | 2d optimal path planning algorithm for autonomous underwater vehicle driving in unknown underwater canyons |
| topic | autonomous underwater vehicle 2D optimal path planning deep reinforcement learning unknown underwater canyons environment |
| url | https://www.mdpi.com/2077-1312/9/3/252 |
| work_keys_str_mv | AT yushansun a2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT xiaokunluo a2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT xiangruiran a2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT guochengzhang a2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT yushansun 2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT xiaokunluo 2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT xiangruiran 2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons AT guochengzhang 2doptimalpathplanningalgorithmforautonomousunderwatervehicledrivinginunknownunderwatercanyons |