iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV
Path planning algorithms are of paramount importance in guidance and collision systems to provide trustworthiness and safety for operations of autonomous unmanned aerial vehicles (UAV). Previous works showed different approaches mostly focusing on shortest path discovery without a sufficient conside...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/9/3948 |
| _version_ | 1797536150474194944 |
|---|---|
| author | Aye Aye Maw Maxim Tyan Tuan Anh Nguyen Jae-Woo Lee |
| author_facet | Aye Aye Maw Maxim Tyan Tuan Anh Nguyen Jae-Woo Lee |
| author_sort | Aye Aye Maw |
| collection | DOAJ |
| description | Path planning algorithms are of paramount importance in guidance and collision systems to provide trustworthiness and safety for operations of autonomous unmanned aerial vehicles (UAV). Previous works showed different approaches mostly focusing on shortest path discovery without a sufficient consideration on local planning and collision avoidance. In this paper, we propose a hybrid path planning algorithm that uses an anytime graph-based path planning algorithm for global planning and deep reinforcement learning for local planning which applied for a real-time mission planning system of an autonomous UAV. In particular, we aim to achieve a highly autonomous UAV mission planning system that is adaptive to real-world environments consisting of both static and moving obstacles for collision avoidance capabilities. To achieve adaptive behavior for real-world problems, a simulator is required that can imitate real environments for learning. For this reason, the simulator must be sufficiently flexible to allow the UAV to learn about the environment and to adapt to real-world conditions. In our scheme, the UAV first learns about the environment via a simulator, and only then is it applied to the real-world. The proposed system is divided into two main parts: optimal flight path generation and collision avoidance. A hybrid path planning approach is developed by combining a graph-based path planning algorithm with a learning-based algorithm for local planning to allow the UAV to avoid a collision in real time. The global path planning problem is solved in the first stage using a novel anytime incremental search algorithm called improved Anytime Dynamic A* (iADA*). A reinforcement learning method is used to carry out local planning between waypoints, to avoid any obstacles within the environment. The developed hybrid path planning system was investigated and validated in an AirSim environment. A number of different simulations and experiments were performed using AirSim platform in order to demonstrate the effectiveness of the proposed system for an autonomous UAV. This study helps expand the existing research area in designing efficient and safe path planning algorithms for UAVs. |
| first_indexed | 2024-03-10T11:55:34Z |
| format | Article |
| id | doaj.art-f82b365632e24676928f5fe2276dbbc0 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T11:55:34Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-f82b365632e24676928f5fe2276dbbc02023-11-21T17:21:14ZengMDPI AGApplied Sciences2076-34172021-04-01119394810.3390/app11093948iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAVAye Aye Maw0Maxim Tyan1Tuan Anh Nguyen2Jae-Woo Lee3Konkuk Aerospace Design-Airworthiness Research Institute (KADA), Konkuk University, Seoul 05029, KoreaKonkuk Aerospace Design-Airworthiness Research Institute (KADA), Konkuk University, Seoul 05029, KoreaKonkuk Aerospace Design-Airworthiness Research Institute (KADA), Konkuk University, Seoul 05029, KoreaDepartment of Aerospace Engineering, Konkuk University, Seoul 05029, KoreaPath planning algorithms are of paramount importance in guidance and collision systems to provide trustworthiness and safety for operations of autonomous unmanned aerial vehicles (UAV). Previous works showed different approaches mostly focusing on shortest path discovery without a sufficient consideration on local planning and collision avoidance. In this paper, we propose a hybrid path planning algorithm that uses an anytime graph-based path planning algorithm for global planning and deep reinforcement learning for local planning which applied for a real-time mission planning system of an autonomous UAV. In particular, we aim to achieve a highly autonomous UAV mission planning system that is adaptive to real-world environments consisting of both static and moving obstacles for collision avoidance capabilities. To achieve adaptive behavior for real-world problems, a simulator is required that can imitate real environments for learning. For this reason, the simulator must be sufficiently flexible to allow the UAV to learn about the environment and to adapt to real-world conditions. In our scheme, the UAV first learns about the environment via a simulator, and only then is it applied to the real-world. The proposed system is divided into two main parts: optimal flight path generation and collision avoidance. A hybrid path planning approach is developed by combining a graph-based path planning algorithm with a learning-based algorithm for local planning to allow the UAV to avoid a collision in real time. The global path planning problem is solved in the first stage using a novel anytime incremental search algorithm called improved Anytime Dynamic A* (iADA*). A reinforcement learning method is used to carry out local planning between waypoints, to avoid any obstacles within the environment. The developed hybrid path planning system was investigated and validated in an AirSim environment. A number of different simulations and experiments were performed using AirSim platform in order to demonstrate the effectiveness of the proposed system for an autonomous UAV. This study helps expand the existing research area in designing efficient and safe path planning algorithms for UAVs.https://www.mdpi.com/2076-3417/11/9/3948reinforcement learninganytime path planning |
| spellingShingle | Aye Aye Maw Maxim Tyan Tuan Anh Nguyen Jae-Woo Lee iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV Applied Sciences reinforcement learning anytime path planning |
| title | iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV |
| title_full | iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV |
| title_fullStr | iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV |
| title_full_unstemmed | iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV |
| title_short | iADA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV |
| title_sort | iada rl anytime graph based path planning with deep reinforcement learning for an autonomous uav |
| topic | reinforcement learning anytime path planning |
| url | https://www.mdpi.com/2076-3417/11/9/3948 |
| work_keys_str_mv | AT ayeayemaw iadarlanytimegraphbasedpathplanningwithdeepreinforcementlearningforanautonomousuav AT maximtyan iadarlanytimegraphbasedpathplanningwithdeepreinforcementlearningforanautonomousuav AT tuananhnguyen iadarlanytimegraphbasedpathplanningwithdeepreinforcementlearningforanautonomousuav AT jaewoolee iadarlanytimegraphbasedpathplanningwithdeepreinforcementlearningforanautonomousuav |