Modified Q-learning with distance metric and virtual target on path planning of mobile robot
Path planning is an essential element in mobile robot navigation. One of the popular path planners is Q-learning – a type of reinforcement learning that learns with little or no prior knowledge of the environment. Despite the successful implementation of Q-learning reported in numerous studies,...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2020
|
| Subjects: | |
| Online Access: | http://eprints.uthm.edu.my/7147/1/J14146_d4792edcf485886c4b01ef6a4fbc4dca.pdf |
| _version_ | 1825710323453132800 |
|---|---|
| author | Ee, Soong Low Ong, Pauline Cheng, Yee Low Omar, Rosli |
| author_facet | Ee, Soong Low Ong, Pauline Cheng, Yee Low Omar, Rosli |
| author_sort | Ee, Soong Low |
| collection | UTHM |
| description | Path planning is an essential element in mobile robot navigation. One of the popular path planners is Q-learning
– a type of reinforcement learning that learns with little or no prior knowledge of the environment. Despite the
successful implementation of Q-learning reported in numerous studies, its slow convergence associated with the
curse of dimensionality may limit the performance in practice. To solve this problem, an Improved Q-learning
(IQL) with three modifications is introduced in this study. First, a distance metric is added to Q-learning to guide
the agent moves towards the target. Second, the Q function of Q-learning is modified to overcome dead-ends
more effectively. Lastly, the virtual target concept is introduced in Q-learning to bypass dead-ends. Experi�mental results across twenty types of navigation maps show that the proposed strategies accelerate the learning
speed of IQL in comparison with the Q-learning. Besides, performance comparison with seven well-known path
planners indicates its efficiency in terms of the path smoothness, time taken, shortest distance and total distance
used. |
| first_indexed | 2024-03-05T21:55:52Z |
| format | Article |
| id | uthm.eprints-7147 |
| institution | Universiti Tun Hussein Onn Malaysia |
| language | English |
| last_indexed | 2024-03-05T21:55:52Z |
| publishDate | 2020 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | uthm.eprints-71472022-06-14T02:08:43Z http://eprints.uthm.edu.my/7147/ Modified Q-learning with distance metric and virtual target on path planning of mobile robot Ee, Soong Low Ong, Pauline Cheng, Yee Low Omar, Rosli T Technology (General) Path planning is an essential element in mobile robot navigation. One of the popular path planners is Q-learning – a type of reinforcement learning that learns with little or no prior knowledge of the environment. Despite the successful implementation of Q-learning reported in numerous studies, its slow convergence associated with the curse of dimensionality may limit the performance in practice. To solve this problem, an Improved Q-learning (IQL) with three modifications is introduced in this study. First, a distance metric is added to Q-learning to guide the agent moves towards the target. Second, the Q function of Q-learning is modified to overcome dead-ends more effectively. Lastly, the virtual target concept is introduced in Q-learning to bypass dead-ends. Experi�mental results across twenty types of navigation maps show that the proposed strategies accelerate the learning speed of IQL in comparison with the Q-learning. Besides, performance comparison with seven well-known path planners indicates its efficiency in terms of the path smoothness, time taken, shortest distance and total distance used. Elsevier 2020 Article PeerReviewed text en http://eprints.uthm.edu.my/7147/1/J14146_d4792edcf485886c4b01ef6a4fbc4dca.pdf Ee, Soong Low and Ong, Pauline and Cheng, Yee Low and Omar, Rosli (2020) Modified Q-learning with distance metric and virtual target on path planning of mobile robot. Expert Systems with Applications, 199. ISSN 0957-4174 https://doi.org/10.1016/j.eswa.2022.117191 |
| spellingShingle | T Technology (General) Ee, Soong Low Ong, Pauline Cheng, Yee Low Omar, Rosli Modified Q-learning with distance metric and virtual target on path planning of mobile robot |
| title | Modified Q-learning with distance metric and virtual target on path planning of mobile robot |
| title_full | Modified Q-learning with distance metric and virtual target on path planning of mobile robot |
| title_fullStr | Modified Q-learning with distance metric and virtual target on path planning of mobile robot |
| title_full_unstemmed | Modified Q-learning with distance metric and virtual target on path planning of mobile robot |
| title_short | Modified Q-learning with distance metric and virtual target on path planning of mobile robot |
| title_sort | modified q learning with distance metric and virtual target on path planning of mobile robot |
| topic | T Technology (General) |
| url | http://eprints.uthm.edu.my/7147/1/J14146_d4792edcf485886c4b01ef6a4fbc4dca.pdf |
| work_keys_str_mv | AT eesoonglow modifiedqlearningwithdistancemetricandvirtualtargetonpathplanningofmobilerobot AT ongpauline modifiedqlearningwithdistancemetricandvirtualtargetonpathplanningofmobilerobot AT chengyeelow modifiedqlearningwithdistancemetricandvirtualtargetonpathplanningofmobilerobot AT omarrosli modifiedqlearningwithdistancemetricandvirtualtargetonpathplanningofmobilerobot |