Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19
Stockpiling and scheduling plans for medical supplies represent essential preventive and control measures in major public health events. In the face of major infectious diseases, such as the novel coronavirus disease (COVID-19), the outbreak trend and variability of disease strains are often unpredi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-12-01
|
| Series: | Operations Research Perspectives |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2214716023000283 |
| _version_ | 1797401773387808768 |
|---|---|
| author | Jia-Ying Zeng Ping Lu Ying Wei Xin Chen Kai-Biao Lin |
| author_facet | Jia-Ying Zeng Ping Lu Ying Wei Xin Chen Kai-Biao Lin |
| author_sort | Jia-Ying Zeng |
| collection | DOAJ |
| description | Stockpiling and scheduling plans for medical supplies represent essential preventive and control measures in major public health events. In the face of major infectious diseases, such as the novel coronavirus disease (COVID-19), the outbreak trend and variability of disease strains are often unpredictable. Hence, it is necessary to optimally adjust the prevention and control dispatching strategy according to the circumstances and outbreak locations to maintain economic development while ensuring the human health survival, however, many models in this scenario seldom consider the dynamic material prediction and the measurement of multiple costs at the same time. Taking the COVID-19 scenario as a case study, we establish a deep reinforcement learning (DRL)-based medical supplies dispatching (MSD) model for major infectious diseases, considering the volatility of the COVID-19 situation and the discrepancy between medical material demand and supply due to the high infectiousness of the Omicron series strains. The present model has three main components: 1) First, for the dynamic medical material prediction problem in complex infectious disease scenarios, taking the lifted COVID-19 lockdown scenario as an example, the modified susceptible-exposed-infected-recovered (SEIR) model was utilized to analyze the spread of the COVID-19, understand its characteristics, and map out the related medical supplies demand; 2) Second, to break away from the previous premise of only considering supply-demand, this study adds scheduling rules and cost function that weighs health and economic costs. An epidemic dispatching optimization model (Epi_DispatchOptim) was established using the OpenAI Gym toolkit to form an environment structure with virus transmission space, and emergency MSD while considering both human health and economic costs. This architecture interprets the balance between the supply-demand of medical supplies and reflects the importance of MSD in the balanced development of health and economy under the spread of infectious diseases; 3) Finally, the MSD strategy under the balance of health and economic cost is explored in Epi_DispatchOptim using reinforcement learning (RL) and the evolutionary algorithm (EA). Experiments conducted on two datasets indicate that the RL and EA reduce economic as well as health costs compared to the original environmental strategies. The above study illustrates how to use epidemiological models to predict the demand for healthcare supplies as the premise of scheduling models, and use Epi_DispatchOptim to explore the dynamic MSD decisions under mortality and economic equilibrium. In Shanghai, China, the economic cost of the exploration strategy is reduced by 27.36–27.07B compared to static scheduling, and deaths are reduced by 126–150 in 150 day compared to the no-intervention scenario. By integrating knowledge of epidemiology, optimal decision making, and economics, Epi_DispatchOptim further constructs epidemiological models, cost functions, state-action spaces, and other modules to assist public health decision makers in adopting appropriate MSD strategies for major public health event. |
| first_indexed | 2024-03-09T02:14:59Z |
| format | Article |
| id | doaj.art-a82385ba1f5240c2810b14c53fdf469c |
| institution | Directory Open Access Journal |
| issn | 2214-7160 |
| language | English |
| last_indexed | 2024-03-09T02:14:59Z |
| publishDate | 2023-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Operations Research Perspectives |
| spelling | doaj.art-a82385ba1f5240c2810b14c53fdf469c2023-12-07T05:28:43ZengElsevierOperations Research Perspectives2214-71602023-12-0111100293Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19Jia-Ying Zeng0Ping Lu1Ying Wei2Xin Chen3Kai-Biao Lin4School of Economic and Management, Xiamen University of Technology, Xiamen 361024, ChinaSchool of Economic and Management, Xiamen University of Technology, Xiamen 361024, China; Corresponding author.Department of Computer Science and Technology, Xiamen University of Technology, Xiamen 361024, ChinaSchool of Data Science and Computer Science, Xiamen Institute of Technology, Xiamen 361021, ChinaDepartment of Computer Science and Technology, Xiamen University of Technology, Xiamen 361024, ChinaStockpiling and scheduling plans for medical supplies represent essential preventive and control measures in major public health events. In the face of major infectious diseases, such as the novel coronavirus disease (COVID-19), the outbreak trend and variability of disease strains are often unpredictable. Hence, it is necessary to optimally adjust the prevention and control dispatching strategy according to the circumstances and outbreak locations to maintain economic development while ensuring the human health survival, however, many models in this scenario seldom consider the dynamic material prediction and the measurement of multiple costs at the same time. Taking the COVID-19 scenario as a case study, we establish a deep reinforcement learning (DRL)-based medical supplies dispatching (MSD) model for major infectious diseases, considering the volatility of the COVID-19 situation and the discrepancy between medical material demand and supply due to the high infectiousness of the Omicron series strains. The present model has three main components: 1) First, for the dynamic medical material prediction problem in complex infectious disease scenarios, taking the lifted COVID-19 lockdown scenario as an example, the modified susceptible-exposed-infected-recovered (SEIR) model was utilized to analyze the spread of the COVID-19, understand its characteristics, and map out the related medical supplies demand; 2) Second, to break away from the previous premise of only considering supply-demand, this study adds scheduling rules and cost function that weighs health and economic costs. An epidemic dispatching optimization model (Epi_DispatchOptim) was established using the OpenAI Gym toolkit to form an environment structure with virus transmission space, and emergency MSD while considering both human health and economic costs. This architecture interprets the balance between the supply-demand of medical supplies and reflects the importance of MSD in the balanced development of health and economy under the spread of infectious diseases; 3) Finally, the MSD strategy under the balance of health and economic cost is explored in Epi_DispatchOptim using reinforcement learning (RL) and the evolutionary algorithm (EA). Experiments conducted on two datasets indicate that the RL and EA reduce economic as well as health costs compared to the original environmental strategies. The above study illustrates how to use epidemiological models to predict the demand for healthcare supplies as the premise of scheduling models, and use Epi_DispatchOptim to explore the dynamic MSD decisions under mortality and economic equilibrium. In Shanghai, China, the economic cost of the exploration strategy is reduced by 27.36–27.07B compared to static scheduling, and deaths are reduced by 126–150 in 150 day compared to the no-intervention scenario. By integrating knowledge of epidemiology, optimal decision making, and economics, Epi_DispatchOptim further constructs epidemiological models, cost functions, state-action spaces, and other modules to assist public health decision makers in adopting appropriate MSD strategies for major public health event.http://www.sciencedirect.com/science/article/pii/S2214716023000283Major public health eventMedical supplies dispatchingDeep reinforcement learningEpidemiological model |
| spellingShingle | Jia-Ying Zeng Ping Lu Ying Wei Xin Chen Kai-Biao Lin Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19 Operations Research Perspectives Major public health event Medical supplies dispatching Deep reinforcement learning Epidemiological model |
| title | Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19 |
| title_full | Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19 |
| title_fullStr | Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19 |
| title_full_unstemmed | Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19 |
| title_short | Deep reinforcement learning based medical supplies dispatching model for major infectious diseases: Case study of COVID-19 |
| title_sort | deep reinforcement learning based medical supplies dispatching model for major infectious diseases case study of covid 19 |
| topic | Major public health event Medical supplies dispatching Deep reinforcement learning Epidemiological model |
| url | http://www.sciencedirect.com/science/article/pii/S2214716023000283 |
| work_keys_str_mv | AT jiayingzeng deepreinforcementlearningbasedmedicalsuppliesdispatchingmodelformajorinfectiousdiseasescasestudyofcovid19 AT pinglu deepreinforcementlearningbasedmedicalsuppliesdispatchingmodelformajorinfectiousdiseasescasestudyofcovid19 AT yingwei deepreinforcementlearningbasedmedicalsuppliesdispatchingmodelformajorinfectiousdiseasescasestudyofcovid19 AT xinchen deepreinforcementlearningbasedmedicalsuppliesdispatchingmodelformajorinfectiousdiseasescasestudyofcovid19 AT kaibiaolin deepreinforcementlearningbasedmedicalsuppliesdispatchingmodelformajorinfectiousdiseasescasestudyofcovid19 |