A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem
Abstract This paper studies the flexible double shop scheduling problem (FDSSP) that considers simultaneously job shop and assembly shop. It brings about the problem of scheduling association of the related tasks. To this end, a reinforcement learning algorithm with a deep temporal difference networ...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-04-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-59414-8 |
| _version_ | 1797199555115089920 |
|---|---|
| author | Xiao Wang Peisi Zhong Mei Liu Chao Zhang Shihao Yang |
| author_facet | Xiao Wang Peisi Zhong Mei Liu Chao Zhang Shihao Yang |
| author_sort | Xiao Wang |
| collection | DOAJ |
| description | Abstract This paper studies the flexible double shop scheduling problem (FDSSP) that considers simultaneously job shop and assembly shop. It brings about the problem of scheduling association of the related tasks. To this end, a reinforcement learning algorithm with a deep temporal difference network is proposed to minimize the makespan. Firstly, the FDSSP is defined as the mathematical model of the flexible job-shop scheduling problem joined to the assembly constraint level. It is translated into a Markov decision process that directly selects behavioral strategies according to historical machining state data. Secondly, the proposed ten generic state features are input into the deep neural network model to fit the state value function. Similarly, eight simple constructive heuristics are used as candidate actions for scheduling decisions. From the greedy mechanism, optimally combined actions of all machines are obtained for each decision step. Finally, a deep temporal difference reinforcement learning framework is established, and a large number of comparative experiments are designed to analyze the basic performance of this algorithm. The results showed that the proposed algorithm was better than most other methods, which contributed to solving the practical production problem of the manufacturing industry. |
| first_indexed | 2024-04-24T07:17:37Z |
| format | Article |
| id | doaj.art-8e5ac27465a4415dab728b02094a4261 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-24T07:17:37Z |
| publishDate | 2024-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-8e5ac27465a4415dab728b02094a42612024-04-21T11:14:48ZengNature PortfolioScientific Reports2045-23222024-04-0114111710.1038/s41598-024-59414-8A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problemXiao Wang0Peisi Zhong1Mei Liu2Chao Zhang3Shihao Yang4College of Mechanical and Electronic Engineering, Shandong University of Science and TechnologyCollege of Mechanical and Electronic Engineering, Shandong University of Science and TechnologyAdvanced Manufacturing Technology Centre, Shandong University of Science and TechnologyCollege of Mechanical and Electronic Engineering, Shandong University of Science and TechnologyCollege of Mechanical and Electronic Engineering, Shandong University of Science and TechnologyAbstract This paper studies the flexible double shop scheduling problem (FDSSP) that considers simultaneously job shop and assembly shop. It brings about the problem of scheduling association of the related tasks. To this end, a reinforcement learning algorithm with a deep temporal difference network is proposed to minimize the makespan. Firstly, the FDSSP is defined as the mathematical model of the flexible job-shop scheduling problem joined to the assembly constraint level. It is translated into a Markov decision process that directly selects behavioral strategies according to historical machining state data. Secondly, the proposed ten generic state features are input into the deep neural network model to fit the state value function. Similarly, eight simple constructive heuristics are used as candidate actions for scheduling decisions. From the greedy mechanism, optimally combined actions of all machines are obtained for each decision step. Finally, a deep temporal difference reinforcement learning framework is established, and a large number of comparative experiments are designed to analyze the basic performance of this algorithm. The results showed that the proposed algorithm was better than most other methods, which contributed to solving the practical production problem of the manufacturing industry.https://doi.org/10.1038/s41598-024-59414-8 |
| spellingShingle | Xiao Wang Peisi Zhong Mei Liu Chao Zhang Shihao Yang A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem Scientific Reports |
| title | A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem |
| title_full | A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem |
| title_fullStr | A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem |
| title_full_unstemmed | A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem |
| title_short | A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem |
| title_sort | novel method based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem |
| url | https://doi.org/10.1038/s41598-024-59414-8 |
| work_keys_str_mv | AT xiaowang anovelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT peisizhong anovelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT meiliu anovelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT chaozhang anovelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT shihaoyang anovelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT xiaowang novelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT peisizhong novelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT meiliu novelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT chaozhang novelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem AT shihaoyang novelmethodbasedreinforcementlearningwithdeeptemporaldifferencenetworkforflexibledoubleshopschedulingproblem |