A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots
The path planning problem has gained more attention due to the gradual popularization of mobile robots. The utilization of reinforcement learning techniques facilitates the ability of mobile robots to successfully navigate through an environment containing obstacles and effectively plan their path....
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Biomimetics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-7673/8/6/481 |
| _version_ | 1797574570747625472 |
|---|---|
| author | Tinglong Zhao Ming Wang Qianchuan Zhao Xuehan Zheng He Gao |
| author_facet | Tinglong Zhao Ming Wang Qianchuan Zhao Xuehan Zheng He Gao |
| author_sort | Tinglong Zhao |
| collection | DOAJ |
| description | The path planning problem has gained more attention due to the gradual popularization of mobile robots. The utilization of reinforcement learning techniques facilitates the ability of mobile robots to successfully navigate through an environment containing obstacles and effectively plan their path. This is achieved by the robots’ interaction with the environment, even in situations when the environment is unfamiliar. Consequently, we provide a refined deep reinforcement learning algorithm that builds upon the soft actor-critic (SAC) algorithm, incorporating the concept of maximum entropy for the purpose of path planning. The objective of this strategy is to mitigate the constraints inherent in conventional reinforcement learning, enhance the efficacy of the learning process, and accommodate intricate situations. In the context of reinforcement learning, two significant issues arise: inadequate incentives and inefficient sample use during the training phase. To address these challenges, the hindsight experience replay (HER) mechanism has been presented as a potential solution. The HER mechanism aims to enhance algorithm performance by effectively reusing past experiences. Through the utilization of simulation studies, it can be demonstrated that the enhanced algorithm exhibits superior performance in comparison with the pre-existing method. |
| first_indexed | 2024-03-10T21:24:21Z |
| format | Article |
| id | doaj.art-4b74132670a344ac87d68ffbebdfbada |
| institution | Directory Open Access Journal |
| issn | 2313-7673 |
| language | English |
| last_indexed | 2024-03-10T21:24:21Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomimetics |
| spelling | doaj.art-4b74132670a344ac87d68ffbebdfbada2023-11-19T15:48:45ZengMDPI AGBiomimetics2313-76732023-10-018648110.3390/biomimetics8060481A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile RobotsTinglong Zhao0Ming Wang1Qianchuan Zhao2Xuehan Zheng3He Gao4School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaSchool of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaDepartment of Automation, Tsinghua University, Beijing 100018, ChinaSchool of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaSchool of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, ChinaThe path planning problem has gained more attention due to the gradual popularization of mobile robots. The utilization of reinforcement learning techniques facilitates the ability of mobile robots to successfully navigate through an environment containing obstacles and effectively plan their path. This is achieved by the robots’ interaction with the environment, even in situations when the environment is unfamiliar. Consequently, we provide a refined deep reinforcement learning algorithm that builds upon the soft actor-critic (SAC) algorithm, incorporating the concept of maximum entropy for the purpose of path planning. The objective of this strategy is to mitigate the constraints inherent in conventional reinforcement learning, enhance the efficacy of the learning process, and accommodate intricate situations. In the context of reinforcement learning, two significant issues arise: inadequate incentives and inefficient sample use during the training phase. To address these challenges, the hindsight experience replay (HER) mechanism has been presented as a potential solution. The HER mechanism aims to enhance algorithm performance by effectively reusing past experiences. Through the utilization of simulation studies, it can be demonstrated that the enhanced algorithm exhibits superior performance in comparison with the pre-existing method.https://www.mdpi.com/2313-7673/8/6/481mobile robotpath planningreinforcement learningsoft actor-critichindsight experience replay |
| spellingShingle | Tinglong Zhao Ming Wang Qianchuan Zhao Xuehan Zheng He Gao A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots Biomimetics mobile robot path planning reinforcement learning soft actor-critic hindsight experience replay |
| title | A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots |
| title_full | A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots |
| title_fullStr | A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots |
| title_full_unstemmed | A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots |
| title_short | A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots |
| title_sort | path planning method based on improved soft actor critic algorithm for mobile robots |
| topic | mobile robot path planning reinforcement learning soft actor-critic hindsight experience replay |
| url | https://www.mdpi.com/2313-7673/8/6/481 |
| work_keys_str_mv | AT tinglongzhao apathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT mingwang apathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT qianchuanzhao apathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT xuehanzheng apathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT hegao apathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT tinglongzhao pathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT mingwang pathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT qianchuanzhao pathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT xuehanzheng pathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots AT hegao pathplanningmethodbasedonimprovedsoftactorcriticalgorithmformobilerobots |