Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking
Uncertainty of target motion, limited perception ability of onboard cameras, and constrained control have brought new challenges to unmanned aerial vehicle (UAV) dynamic target tracking control. In virtue of the powerful fitting ability and learning ability of the neural network, this paper proposes...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Biomimetics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-7673/7/4/197 |
| _version_ | 1797461256937930752 |
|---|---|
| author | Jiang Zhao Han Liu Jiaming Sun Kun Wu Zhihao Cai Yan Ma Yingxun Wang |
| author_facet | Jiang Zhao Han Liu Jiaming Sun Kun Wu Zhihao Cai Yan Ma Yingxun Wang |
| author_sort | Jiang Zhao |
| collection | DOAJ |
| description | Uncertainty of target motion, limited perception ability of onboard cameras, and constrained control have brought new challenges to unmanned aerial vehicle (UAV) dynamic target tracking control. In virtue of the powerful fitting ability and learning ability of the neural network, this paper proposes a new deep reinforcement learning (DRL)-based end-to-end control method for UAV dynamic target tracking. Firstly, a DRL-based framework using onboard camera image is established, which simplifies the traditional modularization paradigm. Secondly, neural network architecture, reward functions, and soft actor-critic (SAC)-based speed command perception algorithm are designed to train the policy network. The output of the policy network is denormalized and directly used as speed control command, which realizes the UAV dynamic target tracking. Finally, the feasibility of the proposed end-to-end control method is demonstrated by numerical simulation. The results show that the proposed DRL-based framework is feasible to simplify the traditional modularization paradigm. The UAV can track the dynamic target with rapidly changing of speed and direction. |
| first_indexed | 2024-03-09T17:16:51Z |
| format | Article |
| id | doaj.art-0085ac9e1d00434fa16bdc77534918e1 |
| institution | Directory Open Access Journal |
| issn | 2313-7673 |
| language | English |
| last_indexed | 2024-03-09T17:16:51Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomimetics |
| spelling | doaj.art-0085ac9e1d00434fa16bdc77534918e12023-11-24T13:31:17ZengMDPI AGBiomimetics2313-76732022-11-017419710.3390/biomimetics7040197Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target TrackingJiang Zhao0Han Liu1Jiaming Sun2Kun Wu3Zhihao Cai4Yan Ma5Yingxun Wang6School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, ChinaSchool of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, ChinaSchool of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, ChinaFlying College, Beihang University, Beijing 100191, ChinaSchool of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, ChinaScience and Technology on Information Systems Engineering Laboratory, Beijing Institute of Control & Electronics Technology, Beijing 100038, ChinaInstitute of Unmanned System, Beihang University, Beijing 100191, ChinaUncertainty of target motion, limited perception ability of onboard cameras, and constrained control have brought new challenges to unmanned aerial vehicle (UAV) dynamic target tracking control. In virtue of the powerful fitting ability and learning ability of the neural network, this paper proposes a new deep reinforcement learning (DRL)-based end-to-end control method for UAV dynamic target tracking. Firstly, a DRL-based framework using onboard camera image is established, which simplifies the traditional modularization paradigm. Secondly, neural network architecture, reward functions, and soft actor-critic (SAC)-based speed command perception algorithm are designed to train the policy network. The output of the policy network is denormalized and directly used as speed control command, which realizes the UAV dynamic target tracking. Finally, the feasibility of the proposed end-to-end control method is demonstrated by numerical simulation. The results show that the proposed DRL-based framework is feasible to simplify the traditional modularization paradigm. The UAV can track the dynamic target with rapidly changing of speed and direction.https://www.mdpi.com/2313-7673/7/4/197unmanned aerial vehicle (UAV)dynamic targettracking controldeep reinforcement learning (DRL)end-to-end controlneural network |
| spellingShingle | Jiang Zhao Han Liu Jiaming Sun Kun Wu Zhihao Cai Yan Ma Yingxun Wang Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking Biomimetics unmanned aerial vehicle (UAV) dynamic target tracking control deep reinforcement learning (DRL) end-to-end control neural network |
| title | Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking |
| title_full | Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking |
| title_fullStr | Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking |
| title_full_unstemmed | Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking |
| title_short | Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking |
| title_sort | deep reinforcement learning based end to end control for uav dynamic target tracking |
| topic | unmanned aerial vehicle (UAV) dynamic target tracking control deep reinforcement learning (DRL) end-to-end control neural network |
| url | https://www.mdpi.com/2313-7673/7/4/197 |
| work_keys_str_mv | AT jiangzhao deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking AT hanliu deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking AT jiamingsun deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking AT kunwu deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking AT zhihaocai deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking AT yanma deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking AT yingxunwang deepreinforcementlearningbasedendtoendcontrolforuavdynamictargettracking |