Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning
Automatic guided vehicles have become an important part of transporting goods in dynamic environments, and how to design an efficient path planning method for multiple AGVs is a current research hotspot. Due to the complex road conditions in dynamic environments, there may be dynamic obstacles and s...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/16/8166 |
| _version_ | 1827623836002025472 |
|---|---|
| author | Yunfei Bai Xuefeng Ding Dasha Hu Yuming Jiang |
| author_facet | Yunfei Bai Xuefeng Ding Dasha Hu Yuming Jiang |
| author_sort | Yunfei Bai |
| collection | DOAJ |
| description | Automatic guided vehicles have become an important part of transporting goods in dynamic environments, and how to design an efficient path planning method for multiple AGVs is a current research hotspot. Due to the complex road conditions in dynamic environments, there may be dynamic obstacles and situations in which only the target point is known but a complete map is lacking, which leads to poor path planning and long planning time for multiple automatic guided vehicles (AGVs). In this paper, a two-level path planning method (referred to as GA-KL, genetic KL method) for multi-AGVs is proposed by integrating the scheduling policy into global path planning and combining the global path planning algorithm and local path planning algorithm. First, for local path planning, we propose an improved Q-learning path optimization algorithm (K-L, Kohonen Q-learning algorithm) based on a Kohonen network, which can avoid dynamic obstacles and complete autonomous path finding using the autonomous learning function of the Q-learning algorithm. Then, we adopt the idea of combining global and local planning by combining the K-L algorithm with the improved genetic algorithm; in addition, we integrate the scheduling policy into global path planning, which can continuously adjust the scheduling policy of multi-AGVs according to changes in the dynamic environment. Finally, through simulation and field experiments, we verified that the K-L algorithm can accomplish autonomous path finding; compared with the traditional path planning algorithm, the algorithm achieved improves results in path length and convergence time with various maps; the convergence time of the algorithm was reduced by about 6.3%, on average, and the path length was reduced by about 4.6%, on average. The experiments also show that the GA-KL method has satisfactory global search capability and can effectively avoid dynamic obstacles. The final experiments also demonstrated that the GA-KL method reduced the total path completion time by an average of 12.6% and the total path length by an average of 8.4% in narrow working environments or highly congested situations, which considerably improved the efficiency of the multi-AGVs. |
| first_indexed | 2024-03-09T11:56:21Z |
| format | Article |
| id | doaj.art-312ffd50a8b94cb192ad1126ddd9bf79 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T11:56:21Z |
| publishDate | 2022-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-312ffd50a8b94cb192ad1126ddd9bf792023-11-30T23:08:16ZengMDPI AGApplied Sciences2076-34172022-08-011216816610.3390/app12168166Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement LearningYunfei Bai0Xuefeng Ding1Dasha Hu2Yuming Jiang3College of Computer Science, Sichuan University, Chengdu 610065, ChinaCollege of Computer Science, Sichuan University, Chengdu 610065, ChinaCollege of Computer Science, Sichuan University, Chengdu 610065, ChinaCollege of Computer Science, Sichuan University, Chengdu 610065, ChinaAutomatic guided vehicles have become an important part of transporting goods in dynamic environments, and how to design an efficient path planning method for multiple AGVs is a current research hotspot. Due to the complex road conditions in dynamic environments, there may be dynamic obstacles and situations in which only the target point is known but a complete map is lacking, which leads to poor path planning and long planning time for multiple automatic guided vehicles (AGVs). In this paper, a two-level path planning method (referred to as GA-KL, genetic KL method) for multi-AGVs is proposed by integrating the scheduling policy into global path planning and combining the global path planning algorithm and local path planning algorithm. First, for local path planning, we propose an improved Q-learning path optimization algorithm (K-L, Kohonen Q-learning algorithm) based on a Kohonen network, which can avoid dynamic obstacles and complete autonomous path finding using the autonomous learning function of the Q-learning algorithm. Then, we adopt the idea of combining global and local planning by combining the K-L algorithm with the improved genetic algorithm; in addition, we integrate the scheduling policy into global path planning, which can continuously adjust the scheduling policy of multi-AGVs according to changes in the dynamic environment. Finally, through simulation and field experiments, we verified that the K-L algorithm can accomplish autonomous path finding; compared with the traditional path planning algorithm, the algorithm achieved improves results in path length and convergence time with various maps; the convergence time of the algorithm was reduced by about 6.3%, on average, and the path length was reduced by about 4.6%, on average. The experiments also show that the GA-KL method has satisfactory global search capability and can effectively avoid dynamic obstacles. The final experiments also demonstrated that the GA-KL method reduced the total path completion time by an average of 12.6% and the total path length by an average of 8.4% in narrow working environments or highly congested situations, which considerably improved the efficiency of the multi-AGVs.https://www.mdpi.com/2076-3417/12/16/8166AGVsQ-learningKohonenadaptive and clustering algorithmspotential energy |
| spellingShingle | Yunfei Bai Xuefeng Ding Dasha Hu Yuming Jiang Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning Applied Sciences AGVs Q-learning Kohonen adaptive and clustering algorithms potential energy |
| title | Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning |
| title_full | Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning |
| title_fullStr | Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning |
| title_full_unstemmed | Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning |
| title_short | Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning |
| title_sort | research on dynamic path planning of multi agvs based on reinforcement learning |
| topic | AGVs Q-learning Kohonen adaptive and clustering algorithms potential energy |
| url | https://www.mdpi.com/2076-3417/12/16/8166 |
| work_keys_str_mv | AT yunfeibai researchondynamicpathplanningofmultiagvsbasedonreinforcementlearning AT xuefengding researchondynamicpathplanningofmultiagvsbasedonreinforcementlearning AT dashahu researchondynamicpathplanningofmultiagvsbasedonreinforcementlearning AT yumingjiang researchondynamicpathplanningofmultiagvsbasedonreinforcementlearning |