Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach
With the rapid development of global trade, ports and terminals are playing an increasingly important role, and automatic guided vehicles (AGVs) have been used as the main carriers performing the loading/unloading operations in automated container terminals. In this paper, we investigate a multi-AGV...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/10/23/4575 |
| _version_ | 1797462709093007360 |
|---|---|
| author | Xiyan Zheng Chengji Liang Yu Wang Jian Shi Gino Lim |
| author_facet | Xiyan Zheng Chengji Liang Yu Wang Jian Shi Gino Lim |
| author_sort | Xiyan Zheng |
| collection | DOAJ |
| description | With the rapid development of global trade, ports and terminals are playing an increasingly important role, and automatic guided vehicles (AGVs) have been used as the main carriers performing the loading/unloading operations in automated container terminals. In this paper, we investigate a multi-AGV dynamic scheduling problem to improve the terminal operational efficiency, considering the sophisticated complexity and uncertainty involved in the port terminal operation. We propose to model the dynamic scheduling of AGVs as a Markov decision process (MDP) with mixed decision rules. Then, we develop a novel adaptive learning algorithm based on a deep Q-network (DQN) to generate the optimal policy. The proposed algorithm is trained based on data obtained from interactions with a simulation environment that reflects the real-world operation of an automated in Shanghai, China. The simulation studies show that, compared with conventional scheduling methods using a heuristic algorithm, i.e., genetic algorithm (GA) and rule-based scheduling, terminal the proposed approach performs better in terms of effectiveness and efficiency. |
| first_indexed | 2024-03-09T17:40:29Z |
| format | Article |
| id | doaj.art-32c09bff3c5b4504ad5a6b241c11aaba |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T17:40:29Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-32c09bff3c5b4504ad5a6b241c11aaba2023-11-24T11:35:52ZengMDPI AGMathematics2227-73902022-12-011023457510.3390/math10234575Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning ApproachXiyan Zheng0Chengji Liang1Yu Wang2Jian Shi3Gino Lim4Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, ChinaInstitute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, ChinaInstitute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, ChinaDepartment of Engineering Technology, University of Houston, Houston, TX 77004, USADepartment of Industrial Engineering, University of Houston, Houston, TX 77004, USAWith the rapid development of global trade, ports and terminals are playing an increasingly important role, and automatic guided vehicles (AGVs) have been used as the main carriers performing the loading/unloading operations in automated container terminals. In this paper, we investigate a multi-AGV dynamic scheduling problem to improve the terminal operational efficiency, considering the sophisticated complexity and uncertainty involved in the port terminal operation. We propose to model the dynamic scheduling of AGVs as a Markov decision process (MDP) with mixed decision rules. Then, we develop a novel adaptive learning algorithm based on a deep Q-network (DQN) to generate the optimal policy. The proposed algorithm is trained based on data obtained from interactions with a simulation environment that reflects the real-world operation of an automated in Shanghai, China. The simulation studies show that, compared with conventional scheduling methods using a heuristic algorithm, i.e., genetic algorithm (GA) and rule-based scheduling, terminal the proposed approach performs better in terms of effectiveness and efficiency.https://www.mdpi.com/2227-7390/10/23/4575Multi-AGV schedulingautomated container terminalmixed decision rulesdeep reinforcement learningsimulation-based algorithm analysis |
| spellingShingle | Xiyan Zheng Chengji Liang Yu Wang Jian Shi Gino Lim Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach Mathematics Multi-AGV scheduling automated container terminal mixed decision rules deep reinforcement learning simulation-based algorithm analysis |
| title | Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach |
| title_full | Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach |
| title_fullStr | Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach |
| title_full_unstemmed | Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach |
| title_short | Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach |
| title_sort | multi agv dynamic scheduling in an automated container terminal a deep reinforcement learning approach |
| topic | Multi-AGV scheduling automated container terminal mixed decision rules deep reinforcement learning simulation-based algorithm analysis |
| url | https://www.mdpi.com/2227-7390/10/23/4575 |
| work_keys_str_mv | AT xiyanzheng multiagvdynamicschedulinginanautomatedcontainerterminaladeepreinforcementlearningapproach AT chengjiliang multiagvdynamicschedulinginanautomatedcontainerterminaladeepreinforcementlearningapproach AT yuwang multiagvdynamicschedulinginanautomatedcontainerterminaladeepreinforcementlearningapproach AT jianshi multiagvdynamicschedulinginanautomatedcontainerterminaladeepreinforcementlearningapproach AT ginolim multiagvdynamicschedulinginanautomatedcontainerterminaladeepreinforcementlearningapproach |