TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH
We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2019-06-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W13/857/2019/isprs-archives-XLII-2-W13-857-2019.pdf |
| _version_ | 1818925152826556416 |
|---|---|
| author | K. A. A. Mustafa N. Botteghi B. Sirmacek M. Poel S. Stramigioli |
| author_facet | K. A. A. Mustafa N. Botteghi B. Sirmacek M. Poel S. Stramigioli |
| author_sort | K. A. A. Mustafa |
| collection | DOAJ |
| description | We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot. |
| first_indexed | 2024-12-20T02:36:41Z |
| format | Article |
| id | doaj.art-7178b74d02b047bab1862d70541b1be6 |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-12-20T02:36:41Z |
| publishDate | 2019-06-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-7178b74d02b047bab1862d70541b1be62022-12-21T19:56:26ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342019-06-01XLII-2-W1385786310.5194/isprs-archives-XLII-2-W13-857-2019TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACHK. A. A. Mustafa0N. Botteghi1B. Sirmacek2M. Poel3S. Stramigioli4Robotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, The NetherlandsRobotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, The NetherlandsRobotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, The NetherlandsData Science, Faculty of Electric Engineering, Mathematics and Computer Science, University of Twente, The NetherlandsRobotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, The NetherlandsWe introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W13/857/2019/isprs-archives-XLII-2-W13-857-2019.pdf |
| spellingShingle | K. A. A. Mustafa N. Botteghi B. Sirmacek M. Poel S. Stramigioli TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH |
| title_full | TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH |
| title_fullStr | TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH |
| title_full_unstemmed | TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH |
| title_short | TOWARDS CONTINUOUS CONTROL FOR MOBILE ROBOT NAVIGATION: A REINFORCEMENT LEARNING AND SLAM BASED APPROACH |
| title_sort | towards continuous control for mobile robot navigation a reinforcement learning and slam based approach |
| url | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W13/857/2019/isprs-archives-XLII-2-W13-857-2019.pdf |
| work_keys_str_mv | AT kaamustafa towardscontinuouscontrolformobilerobotnavigationareinforcementlearningandslambasedapproach AT nbotteghi towardscontinuouscontrolformobilerobotnavigationareinforcementlearningandslambasedapproach AT bsirmacek towardscontinuouscontrolformobilerobotnavigationareinforcementlearningandslambasedapproach AT mpoel towardscontinuouscontrolformobilerobotnavigationareinforcementlearningandslambasedapproach AT sstramigioli towardscontinuouscontrolformobilerobotnavigationareinforcementlearningandslambasedapproach |