Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory
Rapid advances of sensing and cloud technologies transform the manufacturing system into a data-rich environment and make production scheduling increasingly complex. Traditional offline scheduling methods are limited in the ability to handle low-volume-high-mix workorders with diverse design specifi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9305707/ |
| _version_ | 1818639953744101376 |
|---|---|
| author | Tong Zhou Dunbing Tang Haihua Zhu Liping Wang |
| author_facet | Tong Zhou Dunbing Tang Haihua Zhu Liping Wang |
| author_sort | Tong Zhou |
| collection | DOAJ |
| description | Rapid advances of sensing and cloud technologies transform the manufacturing system into a data-rich environment and make production scheduling increasingly complex. Traditional offline scheduling methods are limited in the ability to handle low-volume-high-mix workorders with diverse design specifications. Simulation-based methods show the promise for distributed scheduling of manufacturing jobs but are mostly implemented with historical data and empirical rules in a static manner. Recently, artificial intelligence (AI) algorithms fuel increasing interests to solve dynamic scheduling problems in the manufacturing setting. However, it's difficult to utilize high-dimensional data for production scheduling while considering multiple practical objectives for smart manufacturing (e.g., minimize the makespan, reduce production costs, balance workloads). Therefore, this paper presents a new AI scheduler with composite reward functions for data-driven dynamic scheduling of manufacturing jobs under uncertainty in a smart factory. Internet-enabled sensor networks are deployed in the smart factory to track real-time statuses of workorders, machines, and material handling systems. A novel manufacturing value network is developed to take high-dimensional data as the input and then learn the state-action values for real-time decision making. Based on reinforcement learning (RL), composite rewards help the AI scheduler learn efficiently to achieve multiple objectives for production scheduling in real time. The proposed methodology is evaluated and validated with experimental studies in a smart manufacturing setting. Experimental results show that the new AI scheduler not only improves the multi-objective performance metrics in the production scheduling problem but also effectively copes with unexpected events (e.g., urgent workorders, machine failures) in manufacturing systems. |
| first_indexed | 2024-12-16T23:03:34Z |
| format | Article |
| id | doaj.art-da64f66a12a24da3b5ddb571519312cd |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T23:03:34Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-da64f66a12a24da3b5ddb571519312cd2022-12-21T22:12:38ZengIEEEIEEE Access2169-35362021-01-01975276610.1109/ACCESS.2020.30467849305707Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart FactoryTong Zhou0https://orcid.org/0000-0002-8311-0058Dunbing Tang1https://orcid.org/0000-0002-6144-089XHaihua Zhu2https://orcid.org/0000-0001-5967-8914Liping Wang3https://orcid.org/0000-0002-6618-8435College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaRapid advances of sensing and cloud technologies transform the manufacturing system into a data-rich environment and make production scheduling increasingly complex. Traditional offline scheduling methods are limited in the ability to handle low-volume-high-mix workorders with diverse design specifications. Simulation-based methods show the promise for distributed scheduling of manufacturing jobs but are mostly implemented with historical data and empirical rules in a static manner. Recently, artificial intelligence (AI) algorithms fuel increasing interests to solve dynamic scheduling problems in the manufacturing setting. However, it's difficult to utilize high-dimensional data for production scheduling while considering multiple practical objectives for smart manufacturing (e.g., minimize the makespan, reduce production costs, balance workloads). Therefore, this paper presents a new AI scheduler with composite reward functions for data-driven dynamic scheduling of manufacturing jobs under uncertainty in a smart factory. Internet-enabled sensor networks are deployed in the smart factory to track real-time statuses of workorders, machines, and material handling systems. A novel manufacturing value network is developed to take high-dimensional data as the input and then learn the state-action values for real-time decision making. Based on reinforcement learning (RL), composite rewards help the AI scheduler learn efficiently to achieve multiple objectives for production scheduling in real time. The proposed methodology is evaluated and validated with experimental studies in a smart manufacturing setting. Experimental results show that the new AI scheduler not only improves the multi-objective performance metrics in the production scheduling problem but also effectively copes with unexpected events (e.g., urgent workorders, machine failures) in manufacturing systems.https://ieeexplore.ieee.org/document/9305707/Production schedulingreinforcement learningcomposite rewardsmart factoryneural network |
| spellingShingle | Tong Zhou Dunbing Tang Haihua Zhu Liping Wang Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory IEEE Access Production scheduling reinforcement learning composite reward smart factory neural network |
| title | Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory |
| title_full | Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory |
| title_fullStr | Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory |
| title_full_unstemmed | Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory |
| title_short | Reinforcement Learning With Composite Rewards for Production Scheduling in a Smart Factory |
| title_sort | reinforcement learning with composite rewards for production scheduling in a smart factory |
| topic | Production scheduling reinforcement learning composite reward smart factory neural network |
| url | https://ieeexplore.ieee.org/document/9305707/ |
| work_keys_str_mv | AT tongzhou reinforcementlearningwithcompositerewardsforproductionschedulinginasmartfactory AT dunbingtang reinforcementlearningwithcompositerewardsforproductionschedulinginasmartfactory AT haihuazhu reinforcementlearningwithcompositerewardsforproductionschedulinginasmartfactory AT lipingwang reinforcementlearningwithcompositerewardsforproductionschedulinginasmartfactory |