Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots
Detecting transportation pipeline leakage points within chemical plants is difficult due to complex pathways, multi-dimensional survey points, and highly dynamic scenarios. However, hexapod robots’ maneuverability and adaptability make it an ideal candidate for conducting surveys across different pl...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/7/2061 |
| _version_ | 1797212043919491072 |
|---|---|
| author | Liuhongxu Chen Qibiao Wang Chao Deng Bo Xie Xianguo Tuo Gang Jiang |
| author_facet | Liuhongxu Chen Qibiao Wang Chao Deng Bo Xie Xianguo Tuo Gang Jiang |
| author_sort | Liuhongxu Chen |
| collection | DOAJ |
| description | Detecting transportation pipeline leakage points within chemical plants is difficult due to complex pathways, multi-dimensional survey points, and highly dynamic scenarios. However, hexapod robots’ maneuverability and adaptability make it an ideal candidate for conducting surveys across different planes. The path-planning problem of hexapod robots in multi-dimensional environments is a significant challenge, especially when identifying suitable transition points and planning shorter paths to reach survey points while traversing multi-level environments. This study proposes a Particle Swarm Optimization (PSO)-guided Double Deep Q-Network (DDQN) approach, namely, the PSO-guided DDQN (PG-DDQN) algorithm, for solving this problem. The proposed algorithm incorporates the PSO algorithm to supplant the traditional random selection strategy, and the data obtained from this guided approach are subsequently employed to train the DDQN neural network. The multi-dimensional random environment is abstracted into localized maps comprising current and next level planes. Comparative experiments were performed with PG-DDQN, standard DQN, and standard DDQN to evaluate the algorithm’s performance by using multiple randomly generated localized maps. After testing each iteration, each algorithm obtained the total reward values and completion times. The results demonstrate that PG-DDQN exhibited faster convergence under an equivalent iteration count. Compared with standard DQN and standard DDQN, reductions in path-planning time of at least 33.94% and 42.60%, respectively, were observed, significantly improving the robot’s mobility. Finally, the PG-DDQN algorithm was integrated with sensors onto a hexapod robot, and validation was performed through Gazebo simulations and Experiment. The results show that controlling hexapod robots by applying PG-DDQN provides valuable insights for path planning to reach transportation pipeline leakage points within chemical plants. |
| first_indexed | 2024-04-24T10:36:07Z |
| format | Article |
| id | doaj.art-cf352abca1654fcc8331729687027fd9 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-24T10:36:07Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-cf352abca1654fcc8331729687027fd92024-04-12T13:26:06ZengMDPI AGSensors1424-82202024-03-01247206110.3390/s24072061Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod RobotsLiuhongxu Chen0Qibiao Wang1Chao Deng2Bo Xie3Xianguo Tuo4Gang Jiang5School of Computer Science and Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Computer Science and Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Physics and Electronic Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Physics and Electronic Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Physics and Electronic Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Mechanical and Electrical Engineering, Chengdu University of Technology, Chengdu 610059, ChinaDetecting transportation pipeline leakage points within chemical plants is difficult due to complex pathways, multi-dimensional survey points, and highly dynamic scenarios. However, hexapod robots’ maneuverability and adaptability make it an ideal candidate for conducting surveys across different planes. The path-planning problem of hexapod robots in multi-dimensional environments is a significant challenge, especially when identifying suitable transition points and planning shorter paths to reach survey points while traversing multi-level environments. This study proposes a Particle Swarm Optimization (PSO)-guided Double Deep Q-Network (DDQN) approach, namely, the PSO-guided DDQN (PG-DDQN) algorithm, for solving this problem. The proposed algorithm incorporates the PSO algorithm to supplant the traditional random selection strategy, and the data obtained from this guided approach are subsequently employed to train the DDQN neural network. The multi-dimensional random environment is abstracted into localized maps comprising current and next level planes. Comparative experiments were performed with PG-DDQN, standard DQN, and standard DDQN to evaluate the algorithm’s performance by using multiple randomly generated localized maps. After testing each iteration, each algorithm obtained the total reward values and completion times. The results demonstrate that PG-DDQN exhibited faster convergence under an equivalent iteration count. Compared with standard DQN and standard DDQN, reductions in path-planning time of at least 33.94% and 42.60%, respectively, were observed, significantly improving the robot’s mobility. Finally, the PG-DDQN algorithm was integrated with sensors onto a hexapod robot, and validation was performed through Gazebo simulations and Experiment. The results show that controlling hexapod robots by applying PG-DDQN provides valuable insights for path planning to reach transportation pipeline leakage points within chemical plants.https://www.mdpi.com/1424-8220/24/7/2061hexapod robotpathfindingDDQN algorithm |
| spellingShingle | Liuhongxu Chen Qibiao Wang Chao Deng Bo Xie Xianguo Tuo Gang Jiang Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots Sensors hexapod robot pathfinding DDQN algorithm |
| title | Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots |
| title_full | Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots |
| title_fullStr | Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots |
| title_full_unstemmed | Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots |
| title_short | Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots |
| title_sort | improved double deep q network algorithm applied to multi dimensional environment path planning of hexapod robots |
| topic | hexapod robot pathfinding DDQN algorithm |
| url | https://www.mdpi.com/1424-8220/24/7/2061 |
| work_keys_str_mv | AT liuhongxuchen improveddoubledeepqnetworkalgorithmappliedtomultidimensionalenvironmentpathplanningofhexapodrobots AT qibiaowang improveddoubledeepqnetworkalgorithmappliedtomultidimensionalenvironmentpathplanningofhexapodrobots AT chaodeng improveddoubledeepqnetworkalgorithmappliedtomultidimensionalenvironmentpathplanningofhexapodrobots AT boxie improveddoubledeepqnetworkalgorithmappliedtomultidimensionalenvironmentpathplanningofhexapodrobots AT xianguotuo improveddoubledeepqnetworkalgorithmappliedtomultidimensionalenvironmentpathplanningofhexapodrobots AT gangjiang improveddoubledeepqnetworkalgorithmappliedtomultidimensionalenvironmentpathplanningofhexapodrobots |