Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm
Abstract The demand for autonomous motion control of unmanned aerial vehicles in air combat is boosted as taking the initiative in combat appears more and more crucial. Unmanned aerial vehicles inability to manoeuvre autonomously during air combat that features highly dynamic and uncertain manoeuvre...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-12-01
|
| Series: | CAAI Transactions on Intelligence Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cit2.12195 |
| _version_ | 1797383485428596736 |
|---|---|
| author | Bo Li Shuangxia Bai Shiyang Liang Rui Ma Evgeny Neretin Jingyi Huang |
| author_facet | Bo Li Shuangxia Bai Shiyang Liang Rui Ma Evgeny Neretin Jingyi Huang |
| author_sort | Bo Li |
| collection | DOAJ |
| description | Abstract The demand for autonomous motion control of unmanned aerial vehicles in air combat is boosted as taking the initiative in combat appears more and more crucial. Unmanned aerial vehicles inability to manoeuvre autonomously during air combat that features highly dynamic and uncertain manoeuvres of the enemy; however, limits their combat capabilities, which proves to be very challenging. To meet the challenge, this article proposes an autonomous manoeuvre decision model using an expert actor‐based soft actor critic algorithm that reconstructs empirical replay buffer with expert experience. Specifically, the algorithm uses a small amount of expert experience to increase the diversity of the samples, which can largely improve the exploration and utilisation efficiency of deep reinforcement learning. And to simulate the complex battlefield environment, a one‐to‐one air combat model is established and the concept of missile's attack region is introduced. The model enables the one‐to‐one air combat to be simulated under different initial battlefield situations. Simulation results show that the expert actor‐based soft actor critic algorithm can find the most favourable policy for unmanned aerial vehicles to defeat the opponent faster, and converge more quickly, compared with the soft actor critic algorithm. |
| first_indexed | 2024-03-08T21:21:41Z |
| format | Article |
| id | doaj.art-d043c6de13964da2afd87864c2936020 |
| institution | Directory Open Access Journal |
| issn | 2468-2322 |
| language | English |
| last_indexed | 2024-03-08T21:21:41Z |
| publishDate | 2023-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | CAAI Transactions on Intelligence Technology |
| spelling | doaj.art-d043c6de13964da2afd87864c29360202023-12-21T09:45:30ZengWileyCAAI Transactions on Intelligence Technology2468-23222023-12-01841608161910.1049/cit2.12195Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithmBo Li0Shuangxia Bai1Shiyang Liang2Rui Ma3Evgeny Neretin4Jingyi Huang5School of Electronics and Information Northwestern Polytechnical University Xi'an ChinaSchool of Electronics and Information Northwestern Polytechnical University Xi'an ChinaGeneral department Avic Luoyang Electro‐optical Equipment Research Institute Luoyang ChinaGeneral department Xi'an Electronic Engineering Research Institute Xi'an ChinaSchool of Robotic and Intelligent Systems Moscow Aviation Institute Moscow RussiaSchool of Electronics and Information Northwestern Polytechnical University Xi'an ChinaAbstract The demand for autonomous motion control of unmanned aerial vehicles in air combat is boosted as taking the initiative in combat appears more and more crucial. Unmanned aerial vehicles inability to manoeuvre autonomously during air combat that features highly dynamic and uncertain manoeuvres of the enemy; however, limits their combat capabilities, which proves to be very challenging. To meet the challenge, this article proposes an autonomous manoeuvre decision model using an expert actor‐based soft actor critic algorithm that reconstructs empirical replay buffer with expert experience. Specifically, the algorithm uses a small amount of expert experience to increase the diversity of the samples, which can largely improve the exploration and utilisation efficiency of deep reinforcement learning. And to simulate the complex battlefield environment, a one‐to‐one air combat model is established and the concept of missile's attack region is introduced. The model enables the one‐to‐one air combat to be simulated under different initial battlefield situations. Simulation results show that the expert actor‐based soft actor critic algorithm can find the most favourable policy for unmanned aerial vehicles to defeat the opponent faster, and converge more quickly, compared with the soft actor critic algorithm.https://doi.org/10.1049/cit2.12195artificial intelligence techniquesautonomous dronedeep neural networks |
| spellingShingle | Bo Li Shuangxia Bai Shiyang Liang Rui Ma Evgeny Neretin Jingyi Huang Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm CAAI Transactions on Intelligence Technology artificial intelligence techniques autonomous drone deep neural networks |
| title | Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm |
| title_full | Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm |
| title_fullStr | Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm |
| title_full_unstemmed | Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm |
| title_short | Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm |
| title_sort | manoeuvre decision making of unmanned aerial vehicles in air combat based on an expert actor based soft actor critic algorithm |
| topic | artificial intelligence techniques autonomous drone deep neural networks |
| url | https://doi.org/10.1049/cit2.12195 |
| work_keys_str_mv | AT boli manoeuvredecisionmakingofunmannedaerialvehiclesinaircombatbasedonanexpertactorbasedsoftactorcriticalgorithm AT shuangxiabai manoeuvredecisionmakingofunmannedaerialvehiclesinaircombatbasedonanexpertactorbasedsoftactorcriticalgorithm AT shiyangliang manoeuvredecisionmakingofunmannedaerialvehiclesinaircombatbasedonanexpertactorbasedsoftactorcriticalgorithm AT ruima manoeuvredecisionmakingofunmannedaerialvehiclesinaircombatbasedonanexpertactorbasedsoftactorcriticalgorithm AT evgenyneretin manoeuvredecisionmakingofunmannedaerialvehiclesinaircombatbasedonanexpertactorbasedsoftactorcriticalgorithm AT jingyihuang manoeuvredecisionmakingofunmannedaerialvehiclesinaircombatbasedonanexpertactorbasedsoftactorcriticalgorithm |