Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning
The unmanned surface vehicle (USV) has attracted more and more attention because of its basic ability to perform complex maritime tasks autonomously in constrained environments. However, the level of autonomy of one single USV is still limited, especially when deployed in a dynamic environment to pe...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/18/6942 |
| _version_ | 1827656857158680576 |
|---|---|
| author | Jiayi Wen Shaoman Liu Yejin Lin |
| author_facet | Jiayi Wen Shaoman Liu Yejin Lin |
| author_sort | Jiayi Wen |
| collection | DOAJ |
| description | The unmanned surface vehicle (USV) has attracted more and more attention because of its basic ability to perform complex maritime tasks autonomously in constrained environments. However, the level of autonomy of one single USV is still limited, especially when deployed in a dynamic environment to perform multiple tasks simultaneously. Thus, a multi-USV cooperative approach can be adopted to obtain the desired success rate in the presence of multi-mission objectives. In this paper, we propose a cooperative navigating approach by enabling multiple USVs to automatically avoid dynamic obstacles and allocate target areas. To be specific, we propose a multi-agent deep reinforcement learning (MADRL) approach, i.e., a multi-agent deep deterministic policy gradient (MADDPG), to maximize the autonomy level by jointly optimizing the trajectory of USVs, as well as obstacle avoidance and coordination, which is a complex optimization problem usually solved separately. In contrast to other works, we combined dynamic navigation and area assignment to design a task management system based on the MADDPG learning framework. Finally, the experiments were carried out on the Gym platform to verify the effectiveness of the proposed method. |
| first_indexed | 2024-03-09T22:34:11Z |
| format | Article |
| id | doaj.art-ccd94cba18e949efb005d646637883f9 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T22:34:11Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-ccd94cba18e949efb005d646637883f92023-11-23T18:51:52ZengMDPI AGSensors1424-82202022-09-012218694210.3390/s22186942Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement LearningJiayi Wen0Shaoman Liu1Yejin Lin2Lab of Intelligent Marine Vehicles of DMU, Dalian Maritime University, Dalian 116026, ChinaLab of Intelligent Marine Vehicles of DMU, Dalian Maritime University, Dalian 116026, ChinaLab of Intelligent Marine Vehicles of DMU, Dalian Maritime University, Dalian 116026, ChinaThe unmanned surface vehicle (USV) has attracted more and more attention because of its basic ability to perform complex maritime tasks autonomously in constrained environments. However, the level of autonomy of one single USV is still limited, especially when deployed in a dynamic environment to perform multiple tasks simultaneously. Thus, a multi-USV cooperative approach can be adopted to obtain the desired success rate in the presence of multi-mission objectives. In this paper, we propose a cooperative navigating approach by enabling multiple USVs to automatically avoid dynamic obstacles and allocate target areas. To be specific, we propose a multi-agent deep reinforcement learning (MADRL) approach, i.e., a multi-agent deep deterministic policy gradient (MADDPG), to maximize the autonomy level by jointly optimizing the trajectory of USVs, as well as obstacle avoidance and coordination, which is a complex optimization problem usually solved separately. In contrast to other works, we combined dynamic navigation and area assignment to design a task management system based on the MADDPG learning framework. Finally, the experiments were carried out on the Gym platform to verify the effectiveness of the proposed method.https://www.mdpi.com/1424-8220/22/18/6942USVtrajectory designpolicy gradientmulti-agent deep reinforcement learningmulti-object optimization |
| spellingShingle | Jiayi Wen Shaoman Liu Yejin Lin Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning Sensors USV trajectory design policy gradient multi-agent deep reinforcement learning multi-object optimization |
| title | Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning |
| title_full | Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning |
| title_fullStr | Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning |
| title_full_unstemmed | Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning |
| title_short | Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning |
| title_sort | dynamic navigation and area assignment of multiple usvs based on multi agent deep reinforcement learning |
| topic | USV trajectory design policy gradient multi-agent deep reinforcement learning multi-object optimization |
| url | https://www.mdpi.com/1424-8220/22/18/6942 |
| work_keys_str_mv | AT jiayiwen dynamicnavigationandareaassignmentofmultipleusvsbasedonmultiagentdeepreinforcementlearning AT shaomanliu dynamicnavigationandareaassignmentofmultipleusvsbasedonmultiagentdeepreinforcementlearning AT yejinlin dynamicnavigationandareaassignmentofmultipleusvsbasedonmultiagentdeepreinforcementlearning |