Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM
This paper presents a deep reinforcement learning-based path planning algorithm for the multi-arm robot manipulator when there are both fixed and moving obstacles in the workspace. Considering the problem properties such as high dimensionality and continuous action, the proposed algorithm employs th...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/19/9837 |
| _version_ | 1797480632166645760 |
|---|---|
| author | Kwan-Woo Park MyeongSeop Kim Jung-Su Kim Jae-Han Park |
| author_facet | Kwan-Woo Park MyeongSeop Kim Jung-Su Kim Jae-Han Park |
| author_sort | Kwan-Woo Park |
| collection | DOAJ |
| description | This paper presents a deep reinforcement learning-based path planning algorithm for the multi-arm robot manipulator when there are both fixed and moving obstacles in the workspace. Considering the problem properties such as high dimensionality and continuous action, the proposed algorithm employs the SAC (soft actor-critic). Moreover, in order to predict explicitly the future position of the moving obstacle, LSTM (long short-term memory) is used. The SAC-based path planning algorithm is developed using the LSTM. In order to show the performance of the proposed algorithm, simulation results using GAZEBO and experimental results using real manipulators are presented. The simulation and experiment results show that the success ratio of path generation for arbitrary starting and goal points converges to 100%. It is also confirmed that the LSTM successfully predicts the future position of the obstacle. |
| first_indexed | 2024-03-09T22:02:50Z |
| format | Article |
| id | doaj.art-aff443006a934eb88115f0f003ee6bfa |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T22:02:50Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-aff443006a934eb88115f0f003ee6bfa2023-11-23T19:46:28ZengMDPI AGApplied Sciences2076-34172022-09-011219983710.3390/app12199837Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTMKwan-Woo Park0MyeongSeop Kim1Jung-Su Kim2Jae-Han Park3Research Center for Electrical and Information Technology, Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 01811, KoreaResearch Center for Electrical and Information Technology, Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 01811, KoreaResearch Center for Electrical and Information Technology, Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 01811, KoreaApplied Robot R&D Department, Korea Institute of Industrial Technology (KITECH), Ansan 15588, KoreaThis paper presents a deep reinforcement learning-based path planning algorithm for the multi-arm robot manipulator when there are both fixed and moving obstacles in the workspace. Considering the problem properties such as high dimensionality and continuous action, the proposed algorithm employs the SAC (soft actor-critic). Moreover, in order to predict explicitly the future position of the moving obstacle, LSTM (long short-term memory) is used. The SAC-based path planning algorithm is developed using the LSTM. In order to show the performance of the proposed algorithm, simulation results using GAZEBO and experimental results using real manipulators are presented. The simulation and experiment results show that the success ratio of path generation for arbitrary starting and goal points converges to 100%. It is also confirmed that the LSTM successfully predicts the future position of the obstacle.https://www.mdpi.com/2076-3417/12/19/9837path planningmulti-arm manipulatorsmoving obstaclesreinforcement learningsoft actor-critic (SAC)hindsight experience replay (HER) |
| spellingShingle | Kwan-Woo Park MyeongSeop Kim Jung-Su Kim Jae-Han Park Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM Applied Sciences path planning multi-arm manipulators moving obstacles reinforcement learning soft actor-critic (SAC) hindsight experience replay (HER) |
| title | Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM |
| title_full | Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM |
| title_fullStr | Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM |
| title_full_unstemmed | Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM |
| title_short | Path Planning for Multi-Arm Manipulators Using Soft Actor-Critic Algorithm with Position Prediction of Moving Obstacles via LSTM |
| title_sort | path planning for multi arm manipulators using soft actor critic algorithm with position prediction of moving obstacles via lstm |
| topic | path planning multi-arm manipulators moving obstacles reinforcement learning soft actor-critic (SAC) hindsight experience replay (HER) |
| url | https://www.mdpi.com/2076-3417/12/19/9837 |
| work_keys_str_mv | AT kwanwoopark pathplanningformultiarmmanipulatorsusingsoftactorcriticalgorithmwithpositionpredictionofmovingobstaclesvialstm AT myeongseopkim pathplanningformultiarmmanipulatorsusingsoftactorcriticalgorithmwithpositionpredictionofmovingobstaclesvialstm AT jungsukim pathplanningformultiarmmanipulatorsusingsoftactorcriticalgorithmwithpositionpredictionofmovingobstaclesvialstm AT jaehanpark pathplanningformultiarmmanipulatorsusingsoftactorcriticalgorithmwithpositionpredictionofmovingobstaclesvialstm |