Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning
Swarm control of unmanned aerial vehicles (UAV) has emerged as a challenging research area, primarily attributed to the presence of conflicting behaviors among individual UAVs and the influence of external movement disturbances of UAV swarms. However, limited attention has been drawn to addressing t...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/3/1192 |
| _version_ | 1797318989646397440 |
|---|---|
| author | Xiaorong Zhang Yufeng Wang Wenrui Ding Qing Wang Zhilan Zhang Jun Jia |
| author_facet | Xiaorong Zhang Yufeng Wang Wenrui Ding Qing Wang Zhilan Zhang Jun Jia |
| author_sort | Xiaorong Zhang |
| collection | DOAJ |
| description | Swarm control of unmanned aerial vehicles (UAV) has emerged as a challenging research area, primarily attributed to the presence of conflicting behaviors among individual UAVs and the influence of external movement disturbances of UAV swarms. However, limited attention has been drawn to addressing the fission–fusion motion of UAV swarms for unknown dynamic obstacles, as opposed to static ones. A Bio-inspired Fission–Fusion control and planning via Reinforcement Learning (BiFRL) algorithm for the UAV swarm system is presented, which tackles the problem of fission–fusion behavior in the presence of dynamic obstacles with homing capabilities. Firstly, we found the kinematics models for the UAV and swarm controller, and then we proposed a probabilistic starling-inspired topological interaction that achieves reduced overhead communication and faster local convergence. Next, we develop a self-organized fission–fusion control framework and a fission decision algorithm. When dealing with various situations, the swarm can autonomously re-configure itself by fissioning an optimal number of agents to fulfill the corresponding tasks. Finally, we design a sub-swarm confrontation algorithm for path planning optimized by reinforcement learning, where the sub-swarm can engage in encounters with dynamic obstacles while minimizing energy expenditure. Simulation experiments demonstrate the capability of the UAV swarm system to accomplish self-organized fission–fusion control and planning under different interference scenarios. Moreover, the proposed BiFRL algorithm successfully handles adversarial motion with dynamic obstacles and effectively safeguards the parent swarm. |
| first_indexed | 2024-03-08T04:00:21Z |
| format | Article |
| id | doaj.art-3353b881c6e14a619741b933fef56615 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-08T04:00:21Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-3353b881c6e14a619741b933fef566152024-02-09T15:08:11ZengMDPI AGApplied Sciences2076-34172024-01-01143119210.3390/app14031192Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement LearningXiaorong Zhang0Yufeng Wang1Wenrui Ding2Qing Wang3Zhilan Zhang4Jun Jia5School of Electronic Information Engineering, Beihang University, Beijing 100191, ChinaInstitute of Unmanned System, Beihang University, Beijing 100191, ChinaInstitute of Unmanned System, Beihang University, Beijing 100191, ChinaSchool of Automation Science Electrical Engineering, Beihang University, Beijing 100191, ChinaSchool of Electronic Information Engineering, Beihang University, Beijing 100191, ChinaShanghai Eletro-Mechanical Engineering Institute, Shanghai 201109, ChinaSwarm control of unmanned aerial vehicles (UAV) has emerged as a challenging research area, primarily attributed to the presence of conflicting behaviors among individual UAVs and the influence of external movement disturbances of UAV swarms. However, limited attention has been drawn to addressing the fission–fusion motion of UAV swarms for unknown dynamic obstacles, as opposed to static ones. A Bio-inspired Fission–Fusion control and planning via Reinforcement Learning (BiFRL) algorithm for the UAV swarm system is presented, which tackles the problem of fission–fusion behavior in the presence of dynamic obstacles with homing capabilities. Firstly, we found the kinematics models for the UAV and swarm controller, and then we proposed a probabilistic starling-inspired topological interaction that achieves reduced overhead communication and faster local convergence. Next, we develop a self-organized fission–fusion control framework and a fission decision algorithm. When dealing with various situations, the swarm can autonomously re-configure itself by fissioning an optimal number of agents to fulfill the corresponding tasks. Finally, we design a sub-swarm confrontation algorithm for path planning optimized by reinforcement learning, where the sub-swarm can engage in encounters with dynamic obstacles while minimizing energy expenditure. Simulation experiments demonstrate the capability of the UAV swarm system to accomplish self-organized fission–fusion control and planning under different interference scenarios. Moreover, the proposed BiFRL algorithm successfully handles adversarial motion with dynamic obstacles and effectively safeguards the parent swarm.https://www.mdpi.com/2076-3417/14/3/1192UAV swarmdynamic obstaclesfission–fusion controlstarling-inspired topological interactionreinforcement learning |
| spellingShingle | Xiaorong Zhang Yufeng Wang Wenrui Ding Qing Wang Zhilan Zhang Jun Jia Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning Applied Sciences UAV swarm dynamic obstacles fission–fusion control starling-inspired topological interaction reinforcement learning |
| title | Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning |
| title_full | Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning |
| title_fullStr | Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning |
| title_full_unstemmed | Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning |
| title_short | Bio-Inspired Fission–Fusion Control and Planning of Unmanned Aerial Vehicles Swarm Systems via Reinforcement Learning |
| title_sort | bio inspired fission fusion control and planning of unmanned aerial vehicles swarm systems via reinforcement learning |
| topic | UAV swarm dynamic obstacles fission–fusion control starling-inspired topological interaction reinforcement learning |
| url | https://www.mdpi.com/2076-3417/14/3/1192 |
| work_keys_str_mv | AT xiaorongzhang bioinspiredfissionfusioncontrolandplanningofunmannedaerialvehiclesswarmsystemsviareinforcementlearning AT yufengwang bioinspiredfissionfusioncontrolandplanningofunmannedaerialvehiclesswarmsystemsviareinforcementlearning AT wenruiding bioinspiredfissionfusioncontrolandplanningofunmannedaerialvehiclesswarmsystemsviareinforcementlearning AT qingwang bioinspiredfissionfusioncontrolandplanningofunmannedaerialvehiclesswarmsystemsviareinforcementlearning AT zhilanzhang bioinspiredfissionfusioncontrolandplanningofunmannedaerialvehiclesswarmsystemsviareinforcementlearning AT junjia bioinspiredfissionfusioncontrolandplanningofunmannedaerialvehiclesswarmsystemsviareinforcementlearning |