Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning
This paper presents a multi-agent reinforcement learning (MARL) algorithm to address the scheduling and routing problems of multiple automated guided vehicles (AGVs), with the goal of minimizing overall energy consumption. The proposed algorithm is developed based on the multi-agent deep determinist...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/12/5615 |
| _version_ | 1797592651540725760 |
|---|---|
| author | Xianfeng Ye Zhiyun Deng Yanjun Shi Weiming Shen |
| author_facet | Xianfeng Ye Zhiyun Deng Yanjun Shi Weiming Shen |
| author_sort | Xianfeng Ye |
| collection | DOAJ |
| description | This paper presents a multi-agent reinforcement learning (MARL) algorithm to address the scheduling and routing problems of multiple automated guided vehicles (AGVs), with the goal of minimizing overall energy consumption. The proposed algorithm is developed based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm, with modifications made to the action and state space to fit the setting of AGV activities. While previous studies overlooked the energy efficiency of AGVs, this paper develops a well-designed reward function that helps to optimize the overall energy consumption required to fulfill all tasks. Moreover, we incorporate the e-greedy exploration strategy into the proposed algorithm to balance exploration and exploitation during training, which helps it converge faster and achieve better performance. The proposed MARL algorithm is equipped with carefully selected parameters that aid in avoiding obstacles, speeding up path planning, and achieving minimal energy consumption. To demonstrate the effectiveness of the proposed algorithm, three types of numerical experiments including the ϵ-greedy MADDPG, MADDPG, and Q-Learning methods were conducted. The results show that the proposed algorithm can effectively solve the multi-AGV task assignment and path planning problems, and the energy consumption results show that the planned routes can effectively improve energy efficiency. |
| first_indexed | 2024-03-11T01:57:11Z |
| format | Article |
| id | doaj.art-8c16a05a0123497e88fc37ecb356fd0e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T01:57:11Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-8c16a05a0123497e88fc37ecb356fd0e2023-11-18T12:33:37ZengMDPI AGSensors1424-82202023-06-012312561510.3390/s23125615Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement LearningXianfeng Ye0Zhiyun Deng1Yanjun Shi2Weiming Shen3School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, ChinaSchool of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, ChinaDepartment of Mechanical Engineering, Dalian University of Technology, Dalian 116023, ChinaSchool of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, ChinaThis paper presents a multi-agent reinforcement learning (MARL) algorithm to address the scheduling and routing problems of multiple automated guided vehicles (AGVs), with the goal of minimizing overall energy consumption. The proposed algorithm is developed based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm, with modifications made to the action and state space to fit the setting of AGV activities. While previous studies overlooked the energy efficiency of AGVs, this paper develops a well-designed reward function that helps to optimize the overall energy consumption required to fulfill all tasks. Moreover, we incorporate the e-greedy exploration strategy into the proposed algorithm to balance exploration and exploitation during training, which helps it converge faster and achieve better performance. The proposed MARL algorithm is equipped with carefully selected parameters that aid in avoiding obstacles, speeding up path planning, and achieving minimal energy consumption. To demonstrate the effectiveness of the proposed algorithm, three types of numerical experiments including the ϵ-greedy MADDPG, MADDPG, and Q-Learning methods were conducted. The results show that the proposed algorithm can effectively solve the multi-AGV task assignment and path planning problems, and the energy consumption results show that the planned routes can effectively improve energy efficiency.https://www.mdpi.com/1424-8220/23/12/5615automated guided vehiclesmulti-agent reinforcement learningtask assignmentpath planningenergy consumption |
| spellingShingle | Xianfeng Ye Zhiyun Deng Yanjun Shi Weiming Shen Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning Sensors automated guided vehicles multi-agent reinforcement learning task assignment path planning energy consumption |
| title | Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning |
| title_full | Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning |
| title_fullStr | Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning |
| title_full_unstemmed | Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning |
| title_short | Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning |
| title_sort | toward energy efficient routing of multiple agvs with multi agent reinforcement learning |
| topic | automated guided vehicles multi-agent reinforcement learning task assignment path planning energy consumption |
| url | https://www.mdpi.com/1424-8220/23/12/5615 |
| work_keys_str_mv | AT xianfengye towardenergyefficientroutingofmultipleagvswithmultiagentreinforcementlearning AT zhiyundeng towardenergyefficientroutingofmultipleagvswithmultiagentreinforcementlearning AT yanjunshi towardenergyefficientroutingofmultipleagvswithmultiagentreinforcementlearning AT weimingshen towardenergyefficientroutingofmultipleagvswithmultiagentreinforcementlearning |