Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning
This paper presents a framework to alleviate the Deep Reinforcement Learning (DRL) training data sparsity problem that is present in challenging domains by creating a DRL agent training and vehicle integration methodology. The methodology leverages accessible domains to train an agent to solve navig...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-09-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frobt.2021.739023/full |
| _version_ | 1819071971317514240 |
|---|---|
| author | Reeve Lambert Jianwen Li Li-Fan Wu Nina Mahmoudian |
| author_facet | Reeve Lambert Jianwen Li Li-Fan Wu Nina Mahmoudian |
| author_sort | Reeve Lambert |
| collection | DOAJ |
| description | This paper presents a framework to alleviate the Deep Reinforcement Learning (DRL) training data sparsity problem that is present in challenging domains by creating a DRL agent training and vehicle integration methodology. The methodology leverages accessible domains to train an agent to solve navigational problems such as obstacle avoidance and allows the agent to generalize to challenging and inaccessible domains such as those present in marine environments with minimal further training. This is done by integrating a DRL agent at a high level of vehicle control and leveraging existing path planning and proven low-level control methodologies that are utilized in multiple domains. An autonomy package with a tertiary multilevel controller is developed to enable the DRL agent to interface at the prescribed high control level and thus be separated from vehicle dynamics and environmental constraints. An example Deep Q Network (DQN) employing this methodology for obstacle avoidance is trained in a simulated ground environment, and then its ability to generalize across domains is experimentally validated. Experimental validation utilized a simulated water surface environment and real-world deployment of ground and water robotic platforms. This methodology, when used, shows that it is possible to leverage accessible and data rich domains, such as ground, to effectively develop marine DRL agents for use on Autonomous Surface Vehicle (ASV) navigation. This will allow rapid and iterative agent development without the risk of ASV loss, the cost and logistic overhead of marine deployment, and allow landlocked institutions to develop agents for marine applications. |
| first_indexed | 2024-12-21T17:30:18Z |
| format | Article |
| id | doaj.art-6b42ca4c59de43a9a9d2da7ce975e31e |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-12-21T17:30:18Z |
| publishDate | 2021-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-6b42ca4c59de43a9a9d2da7ce975e31e2022-12-21T18:55:56ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442021-09-01810.3389/frobt.2021.739023739023Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement LearningReeve Lambert0Jianwen Li1Li-Fan Wu2Nina Mahmoudian3MS Student, School of Mechanical Engineering, Purdue University, West Lafayette, IN, United StatesPhD Student, School of Mechanical Engineering, Purdue University, West Lafayette, IN, United StatesPhD Student, School of Mechanical Engineering, Purdue University, West Lafayette, IN, United StatesAssociate Professor, School of Mechanical Engineering, Purdue University, West Lafayette, IN, United StatesThis paper presents a framework to alleviate the Deep Reinforcement Learning (DRL) training data sparsity problem that is present in challenging domains by creating a DRL agent training and vehicle integration methodology. The methodology leverages accessible domains to train an agent to solve navigational problems such as obstacle avoidance and allows the agent to generalize to challenging and inaccessible domains such as those present in marine environments with minimal further training. This is done by integrating a DRL agent at a high level of vehicle control and leveraging existing path planning and proven low-level control methodologies that are utilized in multiple domains. An autonomy package with a tertiary multilevel controller is developed to enable the DRL agent to interface at the prescribed high control level and thus be separated from vehicle dynamics and environmental constraints. An example Deep Q Network (DQN) employing this methodology for obstacle avoidance is trained in a simulated ground environment, and then its ability to generalize across domains is experimentally validated. Experimental validation utilized a simulated water surface environment and real-world deployment of ground and water robotic platforms. This methodology, when used, shows that it is possible to leverage accessible and data rich domains, such as ground, to effectively develop marine DRL agents for use on Autonomous Surface Vehicle (ASV) navigation. This will allow rapid and iterative agent development without the risk of ASV loss, the cost and logistic overhead of marine deployment, and allow landlocked institutions to develop agents for marine applications.https://www.frontiersin.org/articles/10.3389/frobt.2021.739023/fullautonomous surface vehicle (ASV)navigation and controlreinforment learningautonomous vehicle navigationmarine robot navigationcross-domain deep reinforcement learning |
| spellingShingle | Reeve Lambert Jianwen Li Li-Fan Wu Nina Mahmoudian Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning Frontiers in Robotics and AI autonomous surface vehicle (ASV) navigation and control reinforment learning autonomous vehicle navigation marine robot navigation cross-domain deep reinforcement learning |
| title | Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning |
| title_full | Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning |
| title_fullStr | Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning |
| title_full_unstemmed | Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning |
| title_short | Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning |
| title_sort | robust asv navigation through ground to water cross domain deep reinforcement learning |
| topic | autonomous surface vehicle (ASV) navigation and control reinforment learning autonomous vehicle navigation marine robot navigation cross-domain deep reinforcement learning |
| url | https://www.frontiersin.org/articles/10.3389/frobt.2021.739023/full |
| work_keys_str_mv | AT reevelambert robustasvnavigationthroughgroundtowatercrossdomaindeepreinforcementlearning AT jianwenli robustasvnavigationthroughgroundtowatercrossdomaindeepreinforcementlearning AT lifanwu robustasvnavigationthroughgroundtowatercrossdomaindeepreinforcementlearning AT ninamahmoudian robustasvnavigationthroughgroundtowatercrossdomaindeepreinforcementlearning |