Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning
In recent years, sensor components similar to human sensory functions have been rapidly developed in the hardware field, enabling the acquisition of information at a level beyond that of humans, and in the software field, artificial intelligence technology has been utilized to enable cognitive abili...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-09-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1210442/full |
| _version_ | 1797691062712532992 |
|---|---|
| author | HyeokSoo Lee HyeokSoo Lee Jongpil Jeong |
| author_facet | HyeokSoo Lee HyeokSoo Lee Jongpil Jeong |
| author_sort | HyeokSoo Lee |
| collection | DOAJ |
| description | In recent years, sensor components similar to human sensory functions have been rapidly developed in the hardware field, enabling the acquisition of information at a level beyond that of humans, and in the software field, artificial intelligence technology has been utilized to enable cognitive abilities and decision-making such as prediction, analysis, and judgment. These changes are being utilized in various industries and fields. In particular, new hardware and software technologies are being rapidly applied to robotics products, showing a level of performance and completeness that was previously unimaginable. In this paper, we researched the topic of establishing an optimal path plan for autonomous driving using LiDAR sensors and deep reinforcement learning in a workplace without map and grid coordinates for mobile robots, which are widely used in logistics and manufacturing sites. For this purpose, we reviewed the hardware configuration of mobile robots capable of autonomous driving, checked the characteristics of the main core sensors, and investigated the core technologies of autonomous driving. In addition, we reviewed the appropriate deep reinforcement learning algorithm to realize the autonomous driving of mobile robots, defined a deep neural network for autonomous driving data conversion, and defined a reward function for path planning. The contents investigated in this paper were built into a simulation environment to verify the autonomous path planning through experiment, and an additional reward technique “Velocity Range-based Evaluation Method” was proposed for further improvement of performance indicators required in the real field, and the effectiveness was verified. The simulation environment and detailed results of experiments are described in this paper, and it is expected as guidance and reference research for applying these technologies in the field. |
| first_indexed | 2024-03-12T02:09:04Z |
| format | Article |
| id | doaj.art-4c0d588e7d24490e922647e15fb4fced |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-03-12T02:09:04Z |
| publishDate | 2023-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-4c0d588e7d24490e922647e15fb4fced2023-09-06T17:41:50ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182023-09-011710.3389/fnbot.2023.12104421210442Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learningHyeokSoo Lee0HyeokSoo Lee1Jongpil Jeong2Department of Smart Factory Convergence, AI Factory Lab, Sungkyunkwan University, Suwon, Republic of KoreaResearch & Development Team, THiRA-UTECH Co., Ltd., Seoul, Republic of KoreaDepartment of Smart Factory Convergence, AI Factory Lab, Sungkyunkwan University, Suwon, Republic of KoreaIn recent years, sensor components similar to human sensory functions have been rapidly developed in the hardware field, enabling the acquisition of information at a level beyond that of humans, and in the software field, artificial intelligence technology has been utilized to enable cognitive abilities and decision-making such as prediction, analysis, and judgment. These changes are being utilized in various industries and fields. In particular, new hardware and software technologies are being rapidly applied to robotics products, showing a level of performance and completeness that was previously unimaginable. In this paper, we researched the topic of establishing an optimal path plan for autonomous driving using LiDAR sensors and deep reinforcement learning in a workplace without map and grid coordinates for mobile robots, which are widely used in logistics and manufacturing sites. For this purpose, we reviewed the hardware configuration of mobile robots capable of autonomous driving, checked the characteristics of the main core sensors, and investigated the core technologies of autonomous driving. In addition, we reviewed the appropriate deep reinforcement learning algorithm to realize the autonomous driving of mobile robots, defined a deep neural network for autonomous driving data conversion, and defined a reward function for path planning. The contents investigated in this paper were built into a simulation environment to verify the autonomous path planning through experiment, and an additional reward technique “Velocity Range-based Evaluation Method” was proposed for further improvement of performance indicators required in the real field, and the effectiveness was verified. The simulation environment and detailed results of experiments are described in this paper, and it is expected as guidance and reference research for applying these technologies in the field.https://www.frontiersin.org/articles/10.3389/fnbot.2023.1210442/fullautonomous mobile robotdeep reinforcement learningcontinuous actionmap-less navigationSLAMLiDAR |
| spellingShingle | HyeokSoo Lee HyeokSoo Lee Jongpil Jeong Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning Frontiers in Neurorobotics autonomous mobile robot deep reinforcement learning continuous action map-less navigation SLAM LiDAR |
| title | Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning |
| title_full | Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning |
| title_fullStr | Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning |
| title_full_unstemmed | Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning |
| title_short | Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning |
| title_sort | velocity range based reward shaping technique for effective map less navigation with lidar sensor and deep reinforcement learning |
| topic | autonomous mobile robot deep reinforcement learning continuous action map-less navigation SLAM LiDAR |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1210442/full |
| work_keys_str_mv | AT hyeoksoolee velocityrangebasedrewardshapingtechniqueforeffectivemaplessnavigationwithlidarsensoranddeepreinforcementlearning AT hyeoksoolee velocityrangebasedrewardshapingtechniqueforeffectivemaplessnavigationwithlidarsensoranddeepreinforcementlearning AT jongpiljeong velocityrangebasedrewardshapingtechniqueforeffectivemaplessnavigationwithlidarsensoranddeepreinforcementlearning |