Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework
This study proposes a digital twin (DT) application framework that integrates deep reinforcement learning (DRL) algorithms for the dynamic scheduling of crane transportation in workshops. DT is used to construct the connection between the workshop service system, logical simulation environment, 3D v...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/13/6/286 |
| _version_ | 1797486204389687296 |
|---|---|
| author | Zhenyu Xu Daofang Chang Miaomiao Sun Tian Luo |
| author_facet | Zhenyu Xu Daofang Chang Miaomiao Sun Tian Luo |
| author_sort | Zhenyu Xu |
| collection | DOAJ |
| description | This study proposes a digital twin (DT) application framework that integrates deep reinforcement learning (DRL) algorithms for the dynamic scheduling of crane transportation in workshops. DT is used to construct the connection between the workshop service system, logical simulation environment, 3D visualization model and physical workshop, and DRL is used to support the core decision in scheduling. First, the dynamic scheduling problem of crane transportation is constructed as a Markov decision process (MDP), and the corresponding double deep Q-network (DDQN) is designed to interact with the logic simulation environment to complete the offline training of the algorithm. Second, the trained DDQN is embedded into the DT framework, and then connected with the physical workshop and the workshop service system to realize online dynamic crane scheduling based on the real-time states of the workshop. Finally, case studies of crane scheduling under dynamic job arrival and equipment failure scenarios are presented to demonstrate the effectiveness of the proposed framework. The numerical analysis shows that the proposed method is superior to the traditional dynamic scheduling method, and it is also suitable for large-scale problems. |
| first_indexed | 2024-03-09T23:29:54Z |
| format | Article |
| id | doaj.art-1714824e15ca43e788126967d4f01c38 |
| institution | Directory Open Access Journal |
| issn | 2078-2489 |
| language | English |
| last_indexed | 2024-03-09T23:29:54Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj.art-1714824e15ca43e788126967d4f01c382023-11-23T17:09:49ZengMDPI AGInformation2078-24892022-06-0113628610.3390/info13060286Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin FrameworkZhenyu Xu0Daofang Chang1Miaomiao Sun2Tian Luo3Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, ChinaInstitute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, ChinaInstitute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, ChinaSchool of Economics & Management, Shanghai Maritime University, Shanghai 201306, ChinaThis study proposes a digital twin (DT) application framework that integrates deep reinforcement learning (DRL) algorithms for the dynamic scheduling of crane transportation in workshops. DT is used to construct the connection between the workshop service system, logical simulation environment, 3D visualization model and physical workshop, and DRL is used to support the core decision in scheduling. First, the dynamic scheduling problem of crane transportation is constructed as a Markov decision process (MDP), and the corresponding double deep Q-network (DDQN) is designed to interact with the logic simulation environment to complete the offline training of the algorithm. Second, the trained DDQN is embedded into the DT framework, and then connected with the physical workshop and the workshop service system to realize online dynamic crane scheduling based on the real-time states of the workshop. Finally, case studies of crane scheduling under dynamic job arrival and equipment failure scenarios are presented to demonstrate the effectiveness of the proposed framework. The numerical analysis shows that the proposed method is superior to the traditional dynamic scheduling method, and it is also suitable for large-scale problems.https://www.mdpi.com/2078-2489/13/6/286digital twindeep reinforcement learningcranedynamic scheduling |
| spellingShingle | Zhenyu Xu Daofang Chang Miaomiao Sun Tian Luo Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework Information digital twin deep reinforcement learning crane dynamic scheduling |
| title | Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework |
| title_full | Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework |
| title_fullStr | Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework |
| title_full_unstemmed | Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework |
| title_short | Dynamic Scheduling of Crane by Embedding Deep Reinforcement Learning into a Digital Twin Framework |
| title_sort | dynamic scheduling of crane by embedding deep reinforcement learning into a digital twin framework |
| topic | digital twin deep reinforcement learning crane dynamic scheduling |
| url | https://www.mdpi.com/2078-2489/13/6/286 |
| work_keys_str_mv | AT zhenyuxu dynamicschedulingofcranebyembeddingdeepreinforcementlearningintoadigitaltwinframework AT daofangchang dynamicschedulingofcranebyembeddingdeepreinforcementlearningintoadigitaltwinframework AT miaomiaosun dynamicschedulingofcranebyembeddingdeepreinforcementlearningintoadigitaltwinframework AT tianluo dynamicschedulingofcranebyembeddingdeepreinforcementlearningintoadigitaltwinframework |