UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking
The success of robotic, such as UGV systems, largely benefits from the fundamental capability of autonomously finding collision-free path(s) to commit mobile tasks in routinely rough and complicated environments. Optimization of navigation under such circumstance has long been an open problem: 1) to...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2018-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8476521/ |
| _version_ | 1828889985467547648 |
|---|---|
| author | Minggao Wei Song Wang Jinfan Zheng Dan Chen |
| author_facet | Minggao Wei Song Wang Jinfan Zheng Dan Chen |
| author_sort | Minggao Wei |
| collection | DOAJ |
| description | The success of robotic, such as UGV systems, largely benefits from the fundamental capability of autonomously finding collision-free path(s) to commit mobile tasks in routinely rough and complicated environments. Optimization of navigation under such circumstance has long been an open problem: 1) to meet the critical requirements of this task typically including the shortest distance and smoothness and 2) more challengingly, to enable a general solution to track the optimal path in real-time outdoor applications. Aiming at the problem, this study develops a two-tier approach to navigation optimization in terms of path planning and tracking. First, a “rope”model has been designed to mimic the deformation of a path in axial direction under external force and the fixedness of the radial plane to contain a UGV in a collision-free space. Second, a deterministic policy gradient (DPG) algorithm has been trained efficiently on abstracted structures of an arbitrarily derived “rope”to model the controller for tracking the optimal path. The learned policy can be generalized to a variety of scenarios. Experiments have been performed over complicated environments of different types. The results indicate that: 1) the rope model helps in minimizing distance and enhancing smoothness of the path, while guarantees the clearance; 2) the DPG can be modeled quickly (in a couple of minutes on an office desktop) and the model can apply to environments of increasing complexity under the circumstance of external disturbances without the need for tuning parameters; and 3) the DPG-based controller can autonomously adjust the UGV to follow the correct path free of risks by itself. |
| first_indexed | 2024-12-13T12:50:40Z |
| format | Article |
| id | doaj.art-9393e6310a3c43ea91b28528e93b3efc |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T12:50:40Z |
| publishDate | 2018-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-9393e6310a3c43ea91b28528e93b3efc2022-12-21T23:45:20ZengIEEEIEEE Access2169-35362018-01-016578145782510.1109/ACCESS.2018.28727518476521UGV Navigation Optimization Aided by Reinforcement Learning-Based Path TrackingMinggao Wei0Song Wang1Jinfan Zheng2Dan Chen3https://orcid.org/0000-0002-7055-141XNational Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, ChinaNational Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, ChinaNational Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, ChinaNational Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, ChinaThe success of robotic, such as UGV systems, largely benefits from the fundamental capability of autonomously finding collision-free path(s) to commit mobile tasks in routinely rough and complicated environments. Optimization of navigation under such circumstance has long been an open problem: 1) to meet the critical requirements of this task typically including the shortest distance and smoothness and 2) more challengingly, to enable a general solution to track the optimal path in real-time outdoor applications. Aiming at the problem, this study develops a two-tier approach to navigation optimization in terms of path planning and tracking. First, a “rope”model has been designed to mimic the deformation of a path in axial direction under external force and the fixedness of the radial plane to contain a UGV in a collision-free space. Second, a deterministic policy gradient (DPG) algorithm has been trained efficiently on abstracted structures of an arbitrarily derived “rope”to model the controller for tracking the optimal path. The learned policy can be generalized to a variety of scenarios. Experiments have been performed over complicated environments of different types. The results indicate that: 1) the rope model helps in minimizing distance and enhancing smoothness of the path, while guarantees the clearance; 2) the DPG can be modeled quickly (in a couple of minutes on an office desktop) and the model can apply to environments of increasing complexity under the circumstance of external disturbances without the need for tuning parameters; and 3) the DPG-based controller can autonomously adjust the UGV to follow the correct path free of risks by itself.https://ieeexplore.ieee.org/document/8476521/UGV navigationreinforcement learningdeterministic policy gradientpath tracking |
| spellingShingle | Minggao Wei Song Wang Jinfan Zheng Dan Chen UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking IEEE Access UGV navigation reinforcement learning deterministic policy gradient path tracking |
| title | UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking |
| title_full | UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking |
| title_fullStr | UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking |
| title_full_unstemmed | UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking |
| title_short | UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking |
| title_sort | ugv navigation optimization aided by reinforcement learning based path tracking |
| topic | UGV navigation reinforcement learning deterministic policy gradient path tracking |
| url | https://ieeexplore.ieee.org/document/8476521/ |
| work_keys_str_mv | AT minggaowei ugvnavigationoptimizationaidedbyreinforcementlearningbasedpathtracking AT songwang ugvnavigationoptimizationaidedbyreinforcementlearningbasedpathtracking AT jinfanzheng ugvnavigationoptimizationaidedbyreinforcementlearningbasedpathtracking AT danchen ugvnavigationoptimizationaidedbyreinforcementlearningbasedpathtracking |