Deep reinforcement learning-based control model for automatic robot navigation
This report explores the application of deep reinforcement learning (DRL) for robot navigation without pre-constructed maps. Several mainstream DRL models, including DDPG, PPO, and TD3, were tested in a simple static obstacle environment, and TD3 was found to have the best performance. The report th...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/168320 |
| _version_ | 1811691540746076160 |
|---|---|
| author | Deng, Haoyuan |
| author2 | Jiang Xudong |
| author_facet | Jiang Xudong Deng, Haoyuan |
| author_sort | Deng, Haoyuan |
| collection | NTU |
| description | This report explores the application of deep reinforcement learning (DRL) for robot navigation without pre-constructed maps. Several mainstream DRL models, including DDPG, PPO, and TD3, were tested in a simple static obstacle environment, and TD3 was found to have the best performance. The report then investigates the incentive effect of different reward values on TD3 training and shows that a slightly increased positive reward value can substantially improve convergence and motivate the robot to reach the best convergence with less time and fewer steps. Additionally, a novel training approach using a pre-trained model from a static environment was proposed, resulting in faster convergence and larger cumulative reward values in a dynamic obstacle environment. However, the method does not perform well in more complex environments, highlighting the need for further optimization of the model structure and feature extraction capabilities. Overall, this report provides important insights into the use of DRL for map-free robot navigation and highlights potential directions for future research. |
| first_indexed | 2024-10-01T06:21:31Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/168320 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:21:31Z |
| publishDate | 2023 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1683202023-07-07T19:36:12Z Deep reinforcement learning-based control model for automatic robot navigation Deng, Haoyuan Jiang Xudong School of Electrical and Electronic Engineering EXDJiang@ntu.edu.sg Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics This report explores the application of deep reinforcement learning (DRL) for robot navigation without pre-constructed maps. Several mainstream DRL models, including DDPG, PPO, and TD3, were tested in a simple static obstacle environment, and TD3 was found to have the best performance. The report then investigates the incentive effect of different reward values on TD3 training and shows that a slightly increased positive reward value can substantially improve convergence and motivate the robot to reach the best convergence with less time and fewer steps. Additionally, a novel training approach using a pre-trained model from a static environment was proposed, resulting in faster convergence and larger cumulative reward values in a dynamic obstacle environment. However, the method does not perform well in more complex environments, highlighting the need for further optimization of the model structure and feature extraction capabilities. Overall, this report provides important insights into the use of DRL for map-free robot navigation and highlights potential directions for future research. Bachelor of Engineering (Electrical and Electronic Engineering) 2023-06-12T02:32:23Z 2023-06-12T02:32:23Z 2023 Final Year Project (FYP) Deng, H. (2023). Deep reinforcement learning-based control model for automatic robot navigation. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/168320 https://hdl.handle.net/10356/168320 en application/pdf application/octet-stream Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics Deng, Haoyuan Deep reinforcement learning-based control model for automatic robot navigation |
| title | Deep reinforcement learning-based control model for automatic robot navigation |
| title_full | Deep reinforcement learning-based control model for automatic robot navigation |
| title_fullStr | Deep reinforcement learning-based control model for automatic robot navigation |
| title_full_unstemmed | Deep reinforcement learning-based control model for automatic robot navigation |
| title_short | Deep reinforcement learning-based control model for automatic robot navigation |
| title_sort | deep reinforcement learning based control model for automatic robot navigation |
| topic | Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics |
| url | https://hdl.handle.net/10356/168320 |
| work_keys_str_mv | AT denghaoyuan deepreinforcementlearningbasedcontrolmodelforautomaticrobotnavigation |