A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients
Abstract Background A growing body of research suggests that the use of computerized decision support systems can better guide disease treatment and reduce the use of social and medical resources. Artificial intelligence (AI) technology is increasingly being used in medical decision-making systems t...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-05-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12911-023-02175-7 |
| _version_ | 1797832053901754368 |
|---|---|
| author | Tianlai Lin Xinjue Zhang Jianbing Gong Rundong Tan Weiming Li Lijun Wang Yingxia Pan Xiang Xu Junhui Gao |
| author_facet | Tianlai Lin Xinjue Zhang Jianbing Gong Rundong Tan Weiming Li Lijun Wang Yingxia Pan Xiang Xu Junhui Gao |
| author_sort | Tianlai Lin |
| collection | DOAJ |
| description | Abstract Background A growing body of research suggests that the use of computerized decision support systems can better guide disease treatment and reduce the use of social and medical resources. Artificial intelligence (AI) technology is increasingly being used in medical decision-making systems to obtain optimal dosing combinations and improve the survival rate of sepsis patients. To meet the real-world requirements of medical applications and make the training model more robust, we replaced the core algorithm applied in an AI-based medical decision support system developed by research teams at the Massachusetts Institute of Technology (MIT) and IMPERIAL College London (ICL) with the deep deterministic policy gradient (DDPG) algorithm. The main objective of this study was to develop an AI-based medical decision-making system that makes decisions closer to those of professional human clinicians and effectively reduces the mortality rate of sepsis patients. Methods We used the same public intensive care unit (ICU) dataset applied by the research teams at MIT and ICL, i.e., the Multiparameter Intelligent Monitoring in Intensive Care III (MIMIC-III) dataset, which contains information on the hospitalizations of 38,600 adult sepsis patients over the age of 15. We applied the DDPG algorithm as a strategy-based reinforcement learning approach to construct an AI-based medical decision-making system and analyzed the model results within a two-dimensional space to obtain the optimal dosing combination decision for sepsis patients. Results The results show that when the clinician administered the exact same dose as that recommended by the AI model, the mortality of the patients reached the lowest rate at 11.59%. At the same time, according to the database, the baseline mortality rate of the patients was calculated as 15.7%. This indicates that the patient mortality rate when difference between the doses administered by clinicians and those determined by the AI model was zero was approximately 4.2% lower than the baseline patient mortality rate found in the dataset. The results also illustrate that when a clinician administered a different dose than that recommended by the AI model, the patient mortality rate increased, and the greater the difference in dose, the higher the patient mortality rate. Furthermore, compared with the medical decision-making system based on the Deep-Q Learning Network (DQN) algorithm developed by the research teams at MIT and ICL, the optimal dosing combination recommended by our model is closer to that given by professional clinicians. Specifically, the number of patient samples administered by clinicians with the exact same dose recommended by our AI model increased by 142.3% compared with the model based on the DQN algorithm, with a reduction in the patient mortality rate of 2.58%. Conclusions The treatment plan generated by our medical decision-making system based on the DDPG algorithm is closer to that of a professional human clinician with a lower mortality rate in hospitalized sepsis patients, which can better help human clinicians deal with complex conditional changes in sepsis patients in an ICU. Our proposed AI-based medical decision-making system has the potential to provide the best reference dosing combinations for additional drugs. |
| first_indexed | 2024-04-09T14:02:40Z |
| format | Article |
| id | doaj.art-db235a1285c240eaa301300345b19bc7 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-04-09T14:02:40Z |
| publishDate | 2023-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-db235a1285c240eaa301300345b19bc72023-05-07T11:15:05ZengBMCBMC Medical Informatics and Decision Making1472-69472023-05-0123111210.1186/s12911-023-02175-7A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patientsTianlai Lin0Xinjue Zhang1Jianbing Gong2Rundong Tan3Weiming Li4Lijun Wang5Yingxia Pan6Xiang Xu7Junhui Gao8Department of Critical Care Medicine, Quanzhou First Hospital Affiliated to Fujian Medical UniversityShanghai Nuanhe Brain Technology Co., LtdShanghai Biotecan Pharmaceuticals Co., LtdShanghai Nuanhe Brain Technology Co., LtdShanghai Nuanhe Brain Technology Co., LtdShanghai Biotecan Pharmaceuticals Co., LtdShanghai Biotecan Pharmaceuticals Co., LtdBeijing Center for Disease Prevention and ControlShanghai Nuanhe Brain Technology Co., LtdAbstract Background A growing body of research suggests that the use of computerized decision support systems can better guide disease treatment and reduce the use of social and medical resources. Artificial intelligence (AI) technology is increasingly being used in medical decision-making systems to obtain optimal dosing combinations and improve the survival rate of sepsis patients. To meet the real-world requirements of medical applications and make the training model more robust, we replaced the core algorithm applied in an AI-based medical decision support system developed by research teams at the Massachusetts Institute of Technology (MIT) and IMPERIAL College London (ICL) with the deep deterministic policy gradient (DDPG) algorithm. The main objective of this study was to develop an AI-based medical decision-making system that makes decisions closer to those of professional human clinicians and effectively reduces the mortality rate of sepsis patients. Methods We used the same public intensive care unit (ICU) dataset applied by the research teams at MIT and ICL, i.e., the Multiparameter Intelligent Monitoring in Intensive Care III (MIMIC-III) dataset, which contains information on the hospitalizations of 38,600 adult sepsis patients over the age of 15. We applied the DDPG algorithm as a strategy-based reinforcement learning approach to construct an AI-based medical decision-making system and analyzed the model results within a two-dimensional space to obtain the optimal dosing combination decision for sepsis patients. Results The results show that when the clinician administered the exact same dose as that recommended by the AI model, the mortality of the patients reached the lowest rate at 11.59%. At the same time, according to the database, the baseline mortality rate of the patients was calculated as 15.7%. This indicates that the patient mortality rate when difference between the doses administered by clinicians and those determined by the AI model was zero was approximately 4.2% lower than the baseline patient mortality rate found in the dataset. The results also illustrate that when a clinician administered a different dose than that recommended by the AI model, the patient mortality rate increased, and the greater the difference in dose, the higher the patient mortality rate. Furthermore, compared with the medical decision-making system based on the Deep-Q Learning Network (DQN) algorithm developed by the research teams at MIT and ICL, the optimal dosing combination recommended by our model is closer to that given by professional clinicians. Specifically, the number of patient samples administered by clinicians with the exact same dose recommended by our AI model increased by 142.3% compared with the model based on the DQN algorithm, with a reduction in the patient mortality rate of 2.58%. Conclusions The treatment plan generated by our medical decision-making system based on the DDPG algorithm is closer to that of a professional human clinician with a lower mortality rate in hospitalized sepsis patients, which can better help human clinicians deal with complex conditional changes in sepsis patients in an ICU. Our proposed AI-based medical decision-making system has the potential to provide the best reference dosing combinations for additional drugs.https://doi.org/10.1186/s12911-023-02175-7ClinicianModelReinforcement learningSepsis |
| spellingShingle | Tianlai Lin Xinjue Zhang Jianbing Gong Rundong Tan Weiming Li Lijun Wang Yingxia Pan Xiang Xu Junhui Gao A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients BMC Medical Informatics and Decision Making Clinician Model Reinforcement learning Sepsis |
| title | A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients |
| title_full | A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients |
| title_fullStr | A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients |
| title_full_unstemmed | A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients |
| title_short | A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients |
| title_sort | dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients |
| topic | Clinician Model Reinforcement learning Sepsis |
| url | https://doi.org/10.1186/s12911-023-02175-7 |
| work_keys_str_mv | AT tianlailin adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT xinjuezhang adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT jianbinggong adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT rundongtan adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT weimingli adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT lijunwang adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT yingxiapan adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT xiangxu adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT junhuigao adosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT tianlailin dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT xinjuezhang dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT jianbinggong dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT rundongtan dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT weimingli dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT lijunwang dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT yingxiapan dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT xiangxu dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients AT junhuigao dosingstrategymodelofdeepdeterministicpolicygradientalgorithmforsepsispatients |