Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning
The collaborative mission capability of multi-UAV has received more and more attention in recent years as the research on multi-UAV theories and applications has intensified. The artificial intelligence technology integrated into the multi-UAV collaborative decision-making system can effectively imp...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9857929/ |
| _version_ | 1828420289135902720 |
|---|---|
| author | Shaowei Li Yuhong Jia Fan Yang Qingyang Qin Hui Gao Yaoming Zhou |
| author_facet | Shaowei Li Yuhong Jia Fan Yang Qingyang Qin Hui Gao Yaoming Zhou |
| author_sort | Shaowei Li |
| collection | DOAJ |
| description | The collaborative mission capability of multi-UAV has received more and more attention in recent years as the research on multi-UAV theories and applications has intensified. The artificial intelligence technology integrated into the multi-UAV collaborative decision-making system can effectively improve the collaborative mission capability of multi-UAV. We propose a multi-agent reinforcement learning algorithm for multi-UAV collaborative decision-making. Our approach is based on the actor-critic algorithm, where each UAV is treated as an actor that collects data decentralized in the environment. A centralized critic provides evaluation information for each training step during the centralized training of these actors. We introduce a gate recurrent unit in the actor to enable the UAV to make reasonable decisions concerning historical decision information. Moreover, we use an attention mechanism to design the centralized critic, which can achieve better learning in a complex environment. Finally, the algorithm is trained and experimented in a multi-UAV air combat scenario developed in the collaborative decision-making environment. The experimental results show that our approach can learn collaborative decision-making strategies with excellent performance, while convergence performance is better compared to other algorithms. |
| first_indexed | 2024-12-10T15:09:38Z |
| format | Article |
| id | doaj.art-da74bbf055594857b9d5af1cb7248d72 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T15:09:38Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-da74bbf055594857b9d5af1cb7248d722022-12-22T01:43:59ZengIEEEIEEE Access2169-35362022-01-0110913859139610.1109/ACCESS.2022.31990709857929Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement LearningShaowei Li0https://orcid.org/0000-0002-8168-9916Yuhong Jia1Fan Yang2Qingyang Qin3Hui Gao4Yaoming Zhou5https://orcid.org/0000-0003-0410-327XSchool of Aeronautic Science and Engineering, Beihang University, Beijing, ChinaSchool of Aeronautic Science and Engineering, Beihang University, Beijing, ChinaSchool of Aeronautic Science and Engineering, Beihang University, Beijing, ChinaSchool of Aeronautic Science and Engineering, Beihang University, Beijing, ChinaSchool of Aeronautic Science and Engineering, Beihang University, Beijing, ChinaSchool of Aeronautic Science and Engineering, Beihang University, Beijing, ChinaThe collaborative mission capability of multi-UAV has received more and more attention in recent years as the research on multi-UAV theories and applications has intensified. The artificial intelligence technology integrated into the multi-UAV collaborative decision-making system can effectively improve the collaborative mission capability of multi-UAV. We propose a multi-agent reinforcement learning algorithm for multi-UAV collaborative decision-making. Our approach is based on the actor-critic algorithm, where each UAV is treated as an actor that collects data decentralized in the environment. A centralized critic provides evaluation information for each training step during the centralized training of these actors. We introduce a gate recurrent unit in the actor to enable the UAV to make reasonable decisions concerning historical decision information. Moreover, we use an attention mechanism to design the centralized critic, which can achieve better learning in a complex environment. Finally, the algorithm is trained and experimented in a multi-UAV air combat scenario developed in the collaborative decision-making environment. The experimental results show that our approach can learn collaborative decision-making strategies with excellent performance, while convergence performance is better compared to other algorithms.https://ieeexplore.ieee.org/document/9857929/UAVmulti-UAVcollaborative decision-makingmulti-agent reinforcement learning |
| spellingShingle | Shaowei Li Yuhong Jia Fan Yang Qingyang Qin Hui Gao Yaoming Zhou Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning IEEE Access UAV multi-UAV collaborative decision-making multi-agent reinforcement learning |
| title | Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning |
| title_full | Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning |
| title_fullStr | Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning |
| title_full_unstemmed | Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning |
| title_short | Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning |
| title_sort | collaborative decision making method for multi uav based on multiagent reinforcement learning |
| topic | UAV multi-UAV collaborative decision-making multi-agent reinforcement learning |
| url | https://ieeexplore.ieee.org/document/9857929/ |
| work_keys_str_mv | AT shaoweili collaborativedecisionmakingmethodformultiuavbasedonmultiagentreinforcementlearning AT yuhongjia collaborativedecisionmakingmethodformultiuavbasedonmultiagentreinforcementlearning AT fanyang collaborativedecisionmakingmethodformultiuavbasedonmultiagentreinforcementlearning AT qingyangqin collaborativedecisionmakingmethodformultiuavbasedonmultiagentreinforcementlearning AT huigao collaborativedecisionmakingmethodformultiuavbasedonmultiagentreinforcementlearning AT yaomingzhou collaborativedecisionmakingmethodformultiuavbasedonmultiagentreinforcementlearning |