Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs
Ducted-fan tail-sitter unmanned aerial vehicles (UAVs) provide versatility and unique benefits, attracting significant attention in various applications. This study focuses on developing a safe reinforcement learning method for back-transition control between level flight mode and hover mode for duc...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Drones |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-446X/7/5/332 |
| _version_ | 1797600446909513728 |
|---|---|
| author | Yanbo Fu Wenjie Zhao Liu Liu |
| author_facet | Yanbo Fu Wenjie Zhao Liu Liu |
| author_sort | Yanbo Fu |
| collection | DOAJ |
| description | Ducted-fan tail-sitter unmanned aerial vehicles (UAVs) provide versatility and unique benefits, attracting significant attention in various applications. This study focuses on developing a safe reinforcement learning method for back-transition control between level flight mode and hover mode for ducted-fan tail-sitter UAVs. Our method enables transition control with a minimal altitude change and transition time while adhering to the velocity constraint. We employ the Trust Region Policy Optimization, Proximal Policy Optimization with Lagrangian, and Constrained Policy Optimization (CPO) algorithms for controller training, showcasing the superiority of the CPO algorithm and the necessity of the velocity constraint. The transition trajectory achieved using the CPO algorithm closely resembles the optimal trajectory obtained via the well-known GPOPS-II software with the SNOPT solver. Meanwhile, the CPO algorithm also exhibits strong robustness under unknown perturbations of UAV model parameters and wind disturbance. |
| first_indexed | 2024-03-11T03:48:15Z |
| format | Article |
| id | doaj.art-f933d9b8812d4c11b1dbcdb7211e366c |
| institution | Directory Open Access Journal |
| issn | 2504-446X |
| language | English |
| last_indexed | 2024-03-11T03:48:15Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Drones |
| spelling | doaj.art-f933d9b8812d4c11b1dbcdb7211e366c2023-11-18T01:07:27ZengMDPI AGDrones2504-446X2023-05-017533210.3390/drones7050332Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVsYanbo Fu0Wenjie Zhao1Liu Liu2School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, ChinaSchool of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, ChinaSchool of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, ChinaDucted-fan tail-sitter unmanned aerial vehicles (UAVs) provide versatility and unique benefits, attracting significant attention in various applications. This study focuses on developing a safe reinforcement learning method for back-transition control between level flight mode and hover mode for ducted-fan tail-sitter UAVs. Our method enables transition control with a minimal altitude change and transition time while adhering to the velocity constraint. We employ the Trust Region Policy Optimization, Proximal Policy Optimization with Lagrangian, and Constrained Policy Optimization (CPO) algorithms for controller training, showcasing the superiority of the CPO algorithm and the necessity of the velocity constraint. The transition trajectory achieved using the CPO algorithm closely resembles the optimal trajectory obtained via the well-known GPOPS-II software with the SNOPT solver. Meanwhile, the CPO algorithm also exhibits strong robustness under unknown perturbations of UAV model parameters and wind disturbance.https://www.mdpi.com/2504-446X/7/5/332safe reinforcement learningducted fantransition controlunmanned aerial vehicle (UAV) |
| spellingShingle | Yanbo Fu Wenjie Zhao Liu Liu Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs Drones safe reinforcement learning ducted fan transition control unmanned aerial vehicle (UAV) |
| title | Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs |
| title_full | Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs |
| title_fullStr | Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs |
| title_full_unstemmed | Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs |
| title_short | Safe Reinforcement Learning for Transition Control of Ducted-Fan UAVs |
| title_sort | safe reinforcement learning for transition control of ducted fan uavs |
| topic | safe reinforcement learning ducted fan transition control unmanned aerial vehicle (UAV) |
| url | https://www.mdpi.com/2504-446X/7/5/332 |
| work_keys_str_mv | AT yanbofu safereinforcementlearningfortransitioncontrolofductedfanuavs AT wenjiezhao safereinforcementlearningfortransitioncontrolofductedfanuavs AT liuliu safereinforcementlearningfortransitioncontrolofductedfanuavs |