Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods
This article presents two new chemical plume tracing (CPT) algorithms for using on autonomous underwater vehicles (AUVs) to locate hydrothermal vents. We aim to design effective CPT navigation algorithms that direct AUVs to trace emitted hydrothermal plumes to the hydrothermal vent. Traditional CPT...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Journal of Marine Science and Engineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-1312/11/2/366 |
| _version_ | 1797620061145399296 |
|---|---|
| author | Lingxiao Wang Shuo Pang |
| author_facet | Lingxiao Wang Shuo Pang |
| author_sort | Lingxiao Wang |
| collection | DOAJ |
| description | This article presents two new chemical plume tracing (CPT) algorithms for using on autonomous underwater vehicles (AUVs) to locate hydrothermal vents. We aim to design effective CPT navigation algorithms that direct AUVs to trace emitted hydrothermal plumes to the hydrothermal vent. Traditional CPT algorithms can be grouped into two categories, including bio-inspired and engineering-based methods, but they are limited by either search inefficiency in turbulent flow environments or high computational costs. To approach this problem, we design a new CPT algorithm by fusing traditional CPT methods. Specifically, two deep reinforcement learning (RL) algorithms, including double deep Q-network (DDQN) and deep deterministic policy gradient (DDPG), are employed to train a customized deep neural network that dynamically combines two traditional CPT algorithms during the search process. Simulation experiments show that both DDQN- and DDPG-based CPT algorithms achieve a high success rate (>90%) in either laminar or turbulent flow environments. Moreover, compared to traditional moth-inspired method, the averaged search time is improved by 67% for the DDQN- and 44% for the DDPG-based CPT algorithms in turbulent flow environments. |
| first_indexed | 2024-03-11T08:36:06Z |
| format | Article |
| id | doaj.art-ce4b58ebe86849cb913401e06f97977b |
| institution | Directory Open Access Journal |
| issn | 2077-1312 |
| language | English |
| last_indexed | 2024-03-11T08:36:06Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Marine Science and Engineering |
| spelling | doaj.art-ce4b58ebe86849cb913401e06f97977b2023-11-16T21:28:10ZengMDPI AGJournal of Marine Science and Engineering2077-13122023-02-0111236610.3390/jmse11020366Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning MethodsLingxiao Wang0Shuo Pang1Electrical Engineering Department, Louisiana Tech University, Ruston, LA 71272, USADepartment of Electrical Engineering and Computer Science, Embry-Riddle Aeronautical University, Daytona Beach, FL 32114, USAThis article presents two new chemical plume tracing (CPT) algorithms for using on autonomous underwater vehicles (AUVs) to locate hydrothermal vents. We aim to design effective CPT navigation algorithms that direct AUVs to trace emitted hydrothermal plumes to the hydrothermal vent. Traditional CPT algorithms can be grouped into two categories, including bio-inspired and engineering-based methods, but they are limited by either search inefficiency in turbulent flow environments or high computational costs. To approach this problem, we design a new CPT algorithm by fusing traditional CPT methods. Specifically, two deep reinforcement learning (RL) algorithms, including double deep Q-network (DDQN) and deep deterministic policy gradient (DDPG), are employed to train a customized deep neural network that dynamically combines two traditional CPT algorithms during the search process. Simulation experiments show that both DDQN- and DDPG-based CPT algorithms achieve a high success rate (>90%) in either laminar or turbulent flow environments. Moreover, compared to traditional moth-inspired method, the averaged search time is improved by 67% for the DDQN- and 44% for the DDPG-based CPT algorithms in turbulent flow environments.https://www.mdpi.com/2077-1312/11/2/366chemical plume tracingdeep reinforcement learningautonomous underwater vehicles |
| spellingShingle | Lingxiao Wang Shuo Pang Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods Journal of Marine Science and Engineering chemical plume tracing deep reinforcement learning autonomous underwater vehicles |
| title | Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods |
| title_full | Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods |
| title_fullStr | Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods |
| title_full_unstemmed | Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods |
| title_short | Autonomous Underwater Vehicle Based Chemical Plume Tracing via Deep Reinforcement Learning Methods |
| title_sort | autonomous underwater vehicle based chemical plume tracing via deep reinforcement learning methods |
| topic | chemical plume tracing deep reinforcement learning autonomous underwater vehicles |
| url | https://www.mdpi.com/2077-1312/11/2/366 |
| work_keys_str_mv | AT lingxiaowang autonomousunderwatervehiclebasedchemicalplumetracingviadeepreinforcementlearningmethods AT shuopang autonomousunderwatervehiclebasedchemicalplumetracingviadeepreinforcementlearningmethods |