An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics
Many achievements toward unmanned surface vehicles have been made using artificial intelligence theory to assist the decisions of the navigator. In particular, there has been rapid development in autonomous collision avoidance techniques that employ the intelligent algorithm of deep reinforcement le...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Journal of Marine Science and Engineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-1312/11/4/812 |
| _version_ | 1797604822794371072 |
|---|---|
| author | Zhe Sun Yunsheng Fan Guofeng Wang |
| author_facet | Zhe Sun Yunsheng Fan Guofeng Wang |
| author_sort | Zhe Sun |
| collection | DOAJ |
| description | Many achievements toward unmanned surface vehicles have been made using artificial intelligence theory to assist the decisions of the navigator. In particular, there has been rapid development in autonomous collision avoidance techniques that employ the intelligent algorithm of deep reinforcement learning. A novel USV collision avoidance algorithm based on deep reinforcement learning theory for real-time maneuvering is proposed. Many improvements toward the autonomous learning framework are carried out to improve the performance of USV collision avoidance, including prioritized experience replay, noisy network, double learning, and dueling architecture, which can significantly enhance the training effect. Additionally, considering the characteristics of the USV collision avoidance problem, two effective methods to enhance training efficiency are proposed. For better training, considering the international regulations for preventing collisions at sea and USV maneuverability, a complete and reliable USV collision avoidance training system is established, demonstrating an efficient learning process in complex encounter situations. A reward signal system in line with the USV characteristics is designed. Based on the Unity maritime virtual simulation platform, an abundant simulation environment for training and testing is designed. Through detailed analysis, verification, and comparison, the improved algorithm outperforms the pre-improved algorithm in terms of stability, average reward, rules learning, and collision avoidance effect, reducing 26.60% more accumulated course deviation and saving 1.13% more time. |
| first_indexed | 2024-03-11T04:52:13Z |
| format | Article |
| id | doaj.art-a5a862776dce40159b4e15051fc6ab9f |
| institution | Directory Open Access Journal |
| issn | 2077-1312 |
| language | English |
| last_indexed | 2024-03-11T04:52:13Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Marine Science and Engineering |
| spelling | doaj.art-a5a862776dce40159b4e15051fc6ab9f2023-11-17T19:56:24ZengMDPI AGJournal of Marine Science and Engineering2077-13122023-04-0111481210.3390/jmse11040812An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation CharacteristicsZhe Sun0Yunsheng Fan1Guofeng Wang2College of Marine Electrical Engineering, Dalian Maritime University, Dalian 116026, ChinaCollege of Marine Electrical Engineering, Dalian Maritime University, Dalian 116026, ChinaCollege of Marine Electrical Engineering, Dalian Maritime University, Dalian 116026, ChinaMany achievements toward unmanned surface vehicles have been made using artificial intelligence theory to assist the decisions of the navigator. In particular, there has been rapid development in autonomous collision avoidance techniques that employ the intelligent algorithm of deep reinforcement learning. A novel USV collision avoidance algorithm based on deep reinforcement learning theory for real-time maneuvering is proposed. Many improvements toward the autonomous learning framework are carried out to improve the performance of USV collision avoidance, including prioritized experience replay, noisy network, double learning, and dueling architecture, which can significantly enhance the training effect. Additionally, considering the characteristics of the USV collision avoidance problem, two effective methods to enhance training efficiency are proposed. For better training, considering the international regulations for preventing collisions at sea and USV maneuverability, a complete and reliable USV collision avoidance training system is established, demonstrating an efficient learning process in complex encounter situations. A reward signal system in line with the USV characteristics is designed. Based on the Unity maritime virtual simulation platform, an abundant simulation environment for training and testing is designed. Through detailed analysis, verification, and comparison, the improved algorithm outperforms the pre-improved algorithm in terms of stability, average reward, rules learning, and collision avoidance effect, reducing 26.60% more accumulated course deviation and saving 1.13% more time.https://www.mdpi.com/2077-1312/11/4/812unmanned surface vehiclesdeep reinforcement learningautonomous collision avoidanceCOLREGs |
| spellingShingle | Zhe Sun Yunsheng Fan Guofeng Wang An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics Journal of Marine Science and Engineering unmanned surface vehicles deep reinforcement learning autonomous collision avoidance COLREGs |
| title | An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics |
| title_full | An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics |
| title_fullStr | An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics |
| title_full_unstemmed | An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics |
| title_short | An Intelligent Algorithm for USVs Collision Avoidance Based on Deep Reinforcement Learning Approach with Navigation Characteristics |
| title_sort | intelligent algorithm for usvs collision avoidance based on deep reinforcement learning approach with navigation characteristics |
| topic | unmanned surface vehicles deep reinforcement learning autonomous collision avoidance COLREGs |
| url | https://www.mdpi.com/2077-1312/11/4/812 |
| work_keys_str_mv | AT zhesun anintelligentalgorithmforusvscollisionavoidancebasedondeepreinforcementlearningapproachwithnavigationcharacteristics AT yunshengfan anintelligentalgorithmforusvscollisionavoidancebasedondeepreinforcementlearningapproachwithnavigationcharacteristics AT guofengwang anintelligentalgorithmforusvscollisionavoidancebasedondeepreinforcementlearningapproachwithnavigationcharacteristics AT zhesun intelligentalgorithmforusvscollisionavoidancebasedondeepreinforcementlearningapproachwithnavigationcharacteristics AT yunshengfan intelligentalgorithmforusvscollisionavoidancebasedondeepreinforcementlearningapproachwithnavigationcharacteristics AT guofengwang intelligentalgorithmforusvscollisionavoidancebasedondeepreinforcementlearningapproachwithnavigationcharacteristics |