Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach
Autonomous unmanned systems have become an attractive vehicle for a myriad of military and civilian applications. This can be partly attributed to their ability to bring payloads for utility, sensing, and other uses for various applications autonomously. However, a key challenge in realizing autonom...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/7/1523 |
| _version_ | 1797608129570013184 |
|---|---|
| author | Sulemana Nantogma Shangyan Zhang Xuewei Yu Xuyang An Yang Xu |
| author_facet | Sulemana Nantogma Shangyan Zhang Xuewei Yu Xuyang An Yang Xu |
| author_sort | Sulemana Nantogma |
| collection | DOAJ |
| description | Autonomous unmanned systems have become an attractive vehicle for a myriad of military and civilian applications. This can be partly attributed to their ability to bring payloads for utility, sensing, and other uses for various applications autonomously. However, a key challenge in realizing autonomous unmanned systems is the ability to perform complex group missions, which require coordination and collaboration among multiple platforms. This paper presents a cooperative navigating task approach that enables multiple unmanned surface vehicles (multi-USV) to autonomously capture a maneuvering target while avoiding both static and dynamic obstacles. The approach adopts a hybrid multi-agent deep reinforcement learning framework that leverages heuristic mechanisms to guide the group mission learning of the vehicles. Specifically, the proposed framework consists of two stages. In the first stage, navigation subgoal sets are generated based on expert knowledge, and a goal selection heuristic model based on the immune network model is used to select navigation targets during training. Next, the selected goals’ executions are learned using actor-critic proximal policy optimization. The simulation results with multi-USV target capture show that the proposed approach is capable of abstracting and guiding the unmanned vehicle group coordination learning and achieving a generally optimized mission execution. |
| first_indexed | 2024-03-11T05:40:16Z |
| format | Article |
| id | doaj.art-de758c3587944e75b337ee522c074be4 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-11T05:40:16Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-de758c3587944e75b337ee522c074be42023-11-17T16:31:48ZengMDPI AGElectronics2079-92922023-03-01127152310.3390/electronics12071523Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning ApproachSulemana Nantogma0Shangyan Zhang1Xuewei Yu2Xuyang An3Yang Xu4School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaChina North Artificial Intelligence & Innovation Research Institute, Beijing 100072, ChinaChina North Artificial Intelligence & Innovation Research Institute, Beijing 100072, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaAutonomous unmanned systems have become an attractive vehicle for a myriad of military and civilian applications. This can be partly attributed to their ability to bring payloads for utility, sensing, and other uses for various applications autonomously. However, a key challenge in realizing autonomous unmanned systems is the ability to perform complex group missions, which require coordination and collaboration among multiple platforms. This paper presents a cooperative navigating task approach that enables multiple unmanned surface vehicles (multi-USV) to autonomously capture a maneuvering target while avoiding both static and dynamic obstacles. The approach adopts a hybrid multi-agent deep reinforcement learning framework that leverages heuristic mechanisms to guide the group mission learning of the vehicles. Specifically, the proposed framework consists of two stages. In the first stage, navigation subgoal sets are generated based on expert knowledge, and a goal selection heuristic model based on the immune network model is used to select navigation targets during training. Next, the selected goals’ executions are learned using actor-critic proximal policy optimization. The simulation results with multi-USV target capture show that the proposed approach is capable of abstracting and guiding the unmanned vehicle group coordination learning and achieving a generally optimized mission execution.https://www.mdpi.com/2079-9292/12/7/1523multi-agent reinforcement learningmulti-USV systemcooperative controltarget capturedeep RLunmanned systems |
| spellingShingle | Sulemana Nantogma Shangyan Zhang Xuewei Yu Xuyang An Yang Xu Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach Electronics multi-agent reinforcement learning multi-USV system cooperative control target capture deep RL unmanned systems |
| title | Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach |
| title_full | Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach |
| title_fullStr | Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach |
| title_full_unstemmed | Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach |
| title_short | Multi-USV Dynamic Navigation and Target Capture: A Guided Multi-Agent Reinforcement Learning Approach |
| title_sort | multi usv dynamic navigation and target capture a guided multi agent reinforcement learning approach |
| topic | multi-agent reinforcement learning multi-USV system cooperative control target capture deep RL unmanned systems |
| url | https://www.mdpi.com/2079-9292/12/7/1523 |
| work_keys_str_mv | AT sulemananantogma multiusvdynamicnavigationandtargetcaptureaguidedmultiagentreinforcementlearningapproach AT shangyanzhang multiusvdynamicnavigationandtargetcaptureaguidedmultiagentreinforcementlearningapproach AT xueweiyu multiusvdynamicnavigationandtargetcaptureaguidedmultiagentreinforcementlearningapproach AT xuyangan multiusvdynamicnavigationandtargetcaptureaguidedmultiagentreinforcementlearningapproach AT yangxu multiusvdynamicnavigationandtargetcaptureaguidedmultiagentreinforcementlearningapproach |