UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy
Unmanned aerial vehicle (UAV) autonomous tracking and landing is playing an increasingly important role in military and civil applications. In particular, machine learning has been successfully introduced to robotics-related tasks. A novel UAV autonomous tracking and landing approach based on a deep...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/19/5630 |
| _version_ | 1827704951743184896 |
|---|---|
| author | Jingyi Xie Xiaodong Peng Haijiao Wang Wenlong Niu Xiao Zheng |
| author_facet | Jingyi Xie Xiaodong Peng Haijiao Wang Wenlong Niu Xiao Zheng |
| author_sort | Jingyi Xie |
| collection | DOAJ |
| description | Unmanned aerial vehicle (UAV) autonomous tracking and landing is playing an increasingly important role in military and civil applications. In particular, machine learning has been successfully introduced to robotics-related tasks. A novel UAV autonomous tracking and landing approach based on a deep reinforcement learning strategy is presented in this paper, with the aim of dealing with the UAV motion control problem in an unpredictable and harsh environment. Instead of building a prior model and inferring the landing actions based on heuristic rules, a model-free method based on a partially observable Markov decision process (POMDP) is proposed. In the POMDP model, the UAV automatically learns the landing maneuver by an end-to-end neural network, which combines the Deep Deterministic Policy Gradients (DDPG) algorithm and heuristic rules. A Modular Open Robots Simulation Engine (MORSE)-based reinforcement learning framework is designed and validated with a continuous UAV tracking and landing task on a randomly moving platform in high sensor noise and intermittent measurements. The simulation results show that when the moving platform is moving in different trajectories, the average landing success rate of the proposed algorithm is about 10% higher than that of the Proportional-Integral-Derivative (PID) method. As an indirect result, a state-of-the-art deep reinforcement learning-based UAV control method is validated, where the UAV can learn the optimal strategy of a continuously autonomous landing and perform properly in a simulation environment. |
| first_indexed | 2024-03-10T15:53:31Z |
| format | Article |
| id | doaj.art-0a850fb5812e4bb69f648c8d5d0bc574 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T15:53:31Z |
| publishDate | 2020-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-0a850fb5812e4bb69f648c8d5d0bc5742023-11-20T15:50:38ZengMDPI AGSensors1424-82202020-10-012019563010.3390/s20195630UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning StrategyJingyi Xie0Xiaodong Peng1Haijiao Wang2Wenlong Niu3Xiao Zheng4Key Laboratory of Electronics and Information Technology for Space System, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, ChinaKey Laboratory of Electronics and Information Technology for Space System, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, ChinaAlibaba Damo Academy, Hangzhou 311121, ChinaKey Laboratory of Electronics and Information Technology for Space System, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, ChinaKey Laboratory of Electronics and Information Technology for Space System, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, ChinaUnmanned aerial vehicle (UAV) autonomous tracking and landing is playing an increasingly important role in military and civil applications. In particular, machine learning has been successfully introduced to robotics-related tasks. A novel UAV autonomous tracking and landing approach based on a deep reinforcement learning strategy is presented in this paper, with the aim of dealing with the UAV motion control problem in an unpredictable and harsh environment. Instead of building a prior model and inferring the landing actions based on heuristic rules, a model-free method based on a partially observable Markov decision process (POMDP) is proposed. In the POMDP model, the UAV automatically learns the landing maneuver by an end-to-end neural network, which combines the Deep Deterministic Policy Gradients (DDPG) algorithm and heuristic rules. A Modular Open Robots Simulation Engine (MORSE)-based reinforcement learning framework is designed and validated with a continuous UAV tracking and landing task on a randomly moving platform in high sensor noise and intermittent measurements. The simulation results show that when the moving platform is moving in different trajectories, the average landing success rate of the proposed algorithm is about 10% higher than that of the Proportional-Integral-Derivative (PID) method. As an indirect result, a state-of-the-art deep reinforcement learning-based UAV control method is validated, where the UAV can learn the optimal strategy of a continuously autonomous landing and perform properly in a simulation environment.https://www.mdpi.com/1424-8220/20/19/5630quadrotor unmanned aerial vehicledeep reinforcement learningautonomous tracking and landing |
| spellingShingle | Jingyi Xie Xiaodong Peng Haijiao Wang Wenlong Niu Xiao Zheng UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy Sensors quadrotor unmanned aerial vehicle deep reinforcement learning autonomous tracking and landing |
| title | UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy |
| title_full | UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy |
| title_fullStr | UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy |
| title_full_unstemmed | UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy |
| title_short | UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy |
| title_sort | uav autonomous tracking and landing based on deep reinforcement learning strategy |
| topic | quadrotor unmanned aerial vehicle deep reinforcement learning autonomous tracking and landing |
| url | https://www.mdpi.com/1424-8220/20/19/5630 |
| work_keys_str_mv | AT jingyixie uavautonomoustrackingandlandingbasedondeepreinforcementlearningstrategy AT xiaodongpeng uavautonomoustrackingandlandingbasedondeepreinforcementlearningstrategy AT haijiaowang uavautonomoustrackingandlandingbasedondeepreinforcementlearningstrategy AT wenlongniu uavautonomoustrackingandlandingbasedondeepreinforcementlearningstrategy AT xiaozheng uavautonomoustrackingandlandingbasedondeepreinforcementlearningstrategy |