Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network
In emergency rescue missions, rescue teams can use UAVs and efficient path planning strategies to provide flexible rescue services for trapped people, which can improve rescue efficiency and reduce personnel risks. However, since the task environment of UAVs is usually complex, uncertain, and commun...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Drones |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-446X/7/9/572 |
| _version_ | 1797580511789449216 |
|---|---|
| author | Wenshan Wang Guoyin Zhang Qingan Da Dan Lu Yingnan Zhao Sizhao Li Dapeng Lang |
| author_facet | Wenshan Wang Guoyin Zhang Qingan Da Dan Lu Yingnan Zhao Sizhao Li Dapeng Lang |
| author_sort | Wenshan Wang |
| collection | DOAJ |
| description | In emergency rescue missions, rescue teams can use UAVs and efficient path planning strategies to provide flexible rescue services for trapped people, which can improve rescue efficiency and reduce personnel risks. However, since the task environment of UAVs is usually complex, uncertain, and communication-limited, traditional path planning methods may not be able to meet practical needs. In this paper, we introduce a whale optimization algorithm into a deep Q-network and propose a path planning algorithm based on a whale-inspired deep Q-network, which enables UAVs to search for targets faster and safer in uncertain and complex environments. In particular, we first transform the UAV path planning problem into a Markov decision process. Then, we design a comprehensive reward function considering the three factors of path length, obstacle avoidance, and energy consumption. Next, we use the main framework of the deep Q-network to approximate the Q-value function by training a deep neural network. During the training phase, the whale optimization algorithm is introduced for path exploration to generate a richer action decision experience. Finally, experiments show that the proposed algorithm can enable the UAV to autonomously plan a collision-free feasible path in an uncertain environment. And compared with classic reinforcement learning algorithms, the proposed algorithm has a better performance in learning efficiency, path planning success rate, and path length. |
| first_indexed | 2024-03-10T22:52:04Z |
| format | Article |
| id | doaj.art-57d1b29371584b279f90bfc025691c5b |
| institution | Directory Open Access Journal |
| issn | 2504-446X |
| language | English |
| last_indexed | 2024-03-10T22:52:04Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Drones |
| spelling | doaj.art-57d1b29371584b279f90bfc025691c5b2023-11-19T10:17:10ZengMDPI AGDrones2504-446X2023-09-017957210.3390/drones7090572Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-NetworkWenshan Wang0Guoyin Zhang1Qingan Da2Dan Lu3Yingnan Zhao4Sizhao Li5Dapeng Lang6College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaCollege of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaCollege of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaCollege of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaCollege of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaCollege of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaCollege of Computer Science and Technology, Harbin Engineering University, Harbin 150001, ChinaIn emergency rescue missions, rescue teams can use UAVs and efficient path planning strategies to provide flexible rescue services for trapped people, which can improve rescue efficiency and reduce personnel risks. However, since the task environment of UAVs is usually complex, uncertain, and communication-limited, traditional path planning methods may not be able to meet practical needs. In this paper, we introduce a whale optimization algorithm into a deep Q-network and propose a path planning algorithm based on a whale-inspired deep Q-network, which enables UAVs to search for targets faster and safer in uncertain and complex environments. In particular, we first transform the UAV path planning problem into a Markov decision process. Then, we design a comprehensive reward function considering the three factors of path length, obstacle avoidance, and energy consumption. Next, we use the main framework of the deep Q-network to approximate the Q-value function by training a deep neural network. During the training phase, the whale optimization algorithm is introduced for path exploration to generate a richer action decision experience. Finally, experiments show that the proposed algorithm can enable the UAV to autonomously plan a collision-free feasible path in an uncertain environment. And compared with classic reinforcement learning algorithms, the proposed algorithm has a better performance in learning efficiency, path planning success rate, and path length.https://www.mdpi.com/2504-446X/7/9/572deep reinforcement learningdeep Q-networkwhale optimization algorithmmulti-UAVpath planning |
| spellingShingle | Wenshan Wang Guoyin Zhang Qingan Da Dan Lu Yingnan Zhao Sizhao Li Dapeng Lang Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network Drones deep reinforcement learning deep Q-network whale optimization algorithm multi-UAV path planning |
| title | Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network |
| title_full | Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network |
| title_fullStr | Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network |
| title_full_unstemmed | Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network |
| title_short | Multiple Unmanned Aerial Vehicle Autonomous Path Planning Algorithm Based on Whale-Inspired Deep Q-Network |
| title_sort | multiple unmanned aerial vehicle autonomous path planning algorithm based on whale inspired deep q network |
| topic | deep reinforcement learning deep Q-network whale optimization algorithm multi-UAV path planning |
| url | https://www.mdpi.com/2504-446X/7/9/572 |
| work_keys_str_mv | AT wenshanwang multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork AT guoyinzhang multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork AT qinganda multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork AT danlu multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork AT yingnanzhao multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork AT sizhaoli multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork AT dapenglang multipleunmannedaerialvehicleautonomouspathplanningalgorithmbasedonwhaleinspireddeepqnetwork |