Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care
Abstract The aim of this work was to develop and evaluate the reinforcement learning algorithm VentAI, which is able to suggest a dynamically optimized mechanical ventilation regime for critically-ill patients. We built, validated and tested its performance on 11,943 events of volume-controlled mech...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2021-02-01
|
| Series: | npj Digital Medicine |
| Online Access: | https://doi.org/10.1038/s41746-021-00388-6 |
| _version_ | 1797428759527161856 |
|---|---|
| author | Arne Peine Ahmed Hallawa Johannes Bickenbach Guido Dartmann Lejla Begic Fazlic Anke Schmeink Gerd Ascheid Christoph Thiemermann Andreas Schuppert Ryan Kindle Leo Celi Gernot Marx Lukas Martin |
| author_facet | Arne Peine Ahmed Hallawa Johannes Bickenbach Guido Dartmann Lejla Begic Fazlic Anke Schmeink Gerd Ascheid Christoph Thiemermann Andreas Schuppert Ryan Kindle Leo Celi Gernot Marx Lukas Martin |
| author_sort | Arne Peine |
| collection | DOAJ |
| description | Abstract The aim of this work was to develop and evaluate the reinforcement learning algorithm VentAI, which is able to suggest a dynamically optimized mechanical ventilation regime for critically-ill patients. We built, validated and tested its performance on 11,943 events of volume-controlled mechanical ventilation derived from 61,532 distinct ICU admissions and tested it on an independent, secondary dataset (200,859 ICU stays; 25,086 mechanical ventilation events). A patient “data fingerprint” of 44 features was extracted as multidimensional time series in 4-hour time steps. We used a Markov decision process, including a reward system and a Q-learning approach, to find the optimized settings for positive end-expiratory pressure (PEEP), fraction of inspired oxygen (FiO2) and ideal body weight-adjusted tidal volume (Vt). The observed outcome was in-hospital or 90-day mortality. VentAI reached a significantly increased estimated performance return of 83.3 (primary dataset) and 84.1 (secondary dataset) compared to physicians’ standard clinical care (51.1). The number of recommended action changes per mechanically ventilated patient constantly exceeded those of the clinicians. VentAI chose 202.9% more frequently ventilation regimes with lower Vt (5–7.5 mL/kg), but 50.8% less for regimes with higher Vt (7.5–10 mL/kg). VentAI recommended 29.3% more frequently PEEP levels of 5–7 cm H2O and 53.6% more frequently PEEP levels of 7–9 cmH2O. VentAI avoided high (>55%) FiO2 values (59.8% decrease), while preferring the range of 50–55% (140.3% increase). In conclusion, VentAI provides reproducible high performance by dynamically choosing an optimized, individualized ventilation strategy and thus might be of benefit for critically ill patients. |
| first_indexed | 2024-03-09T09:03:31Z |
| format | Article |
| id | doaj.art-cbf3e67d243d4b4c9450953527ffd35b |
| institution | Directory Open Access Journal |
| issn | 2398-6352 |
| language | English |
| last_indexed | 2024-03-09T09:03:31Z |
| publishDate | 2021-02-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Digital Medicine |
| spelling | doaj.art-cbf3e67d243d4b4c9450953527ffd35b2023-12-02T11:04:19ZengNature Portfolionpj Digital Medicine2398-63522021-02-014111210.1038/s41746-021-00388-6Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical careArne Peine0Ahmed Hallawa1Johannes Bickenbach2Guido Dartmann3Lejla Begic Fazlic4Anke Schmeink5Gerd Ascheid6Christoph Thiemermann7Andreas Schuppert8Ryan Kindle9Leo Celi10Gernot Marx11Lukas Martin12Department of Intensive Care and Intermediate Care, University Hospital RWTH AachenDepartment of Intensive Care and Intermediate Care, University Hospital RWTH AachenDepartment of Intensive Care and Intermediate Care, University Hospital RWTH AachenEnvironmental Campus Birkenfeld, Trier University of Applied SciencesEnvironmental Campus Birkenfeld, Trier University of Applied SciencesResearch Area Information Theory and Systematic Design of Communication Systems, RWTH Aachen UniversityChair for Integrated Signal Processing Systems, RWTH Aachen UniversityWilliam Harvey Research Institute, Queen Mary University LondonJoint Research Center for Computational Biomedicine, RWTH Aachen UniversityLaboratory for Computational Physiology, Harvard–MIT Division of Health Sciences & TechnologyLaboratory for Computational Physiology, Harvard–MIT Division of Health Sciences & TechnologyDepartment of Intensive Care and Intermediate Care, University Hospital RWTH AachenDepartment of Intensive Care and Intermediate Care, University Hospital RWTH AachenAbstract The aim of this work was to develop and evaluate the reinforcement learning algorithm VentAI, which is able to suggest a dynamically optimized mechanical ventilation regime for critically-ill patients. We built, validated and tested its performance on 11,943 events of volume-controlled mechanical ventilation derived from 61,532 distinct ICU admissions and tested it on an independent, secondary dataset (200,859 ICU stays; 25,086 mechanical ventilation events). A patient “data fingerprint” of 44 features was extracted as multidimensional time series in 4-hour time steps. We used a Markov decision process, including a reward system and a Q-learning approach, to find the optimized settings for positive end-expiratory pressure (PEEP), fraction of inspired oxygen (FiO2) and ideal body weight-adjusted tidal volume (Vt). The observed outcome was in-hospital or 90-day mortality. VentAI reached a significantly increased estimated performance return of 83.3 (primary dataset) and 84.1 (secondary dataset) compared to physicians’ standard clinical care (51.1). The number of recommended action changes per mechanically ventilated patient constantly exceeded those of the clinicians. VentAI chose 202.9% more frequently ventilation regimes with lower Vt (5–7.5 mL/kg), but 50.8% less for regimes with higher Vt (7.5–10 mL/kg). VentAI recommended 29.3% more frequently PEEP levels of 5–7 cm H2O and 53.6% more frequently PEEP levels of 7–9 cmH2O. VentAI avoided high (>55%) FiO2 values (59.8% decrease), while preferring the range of 50–55% (140.3% increase). In conclusion, VentAI provides reproducible high performance by dynamically choosing an optimized, individualized ventilation strategy and thus might be of benefit for critically ill patients.https://doi.org/10.1038/s41746-021-00388-6 |
| spellingShingle | Arne Peine Ahmed Hallawa Johannes Bickenbach Guido Dartmann Lejla Begic Fazlic Anke Schmeink Gerd Ascheid Christoph Thiemermann Andreas Schuppert Ryan Kindle Leo Celi Gernot Marx Lukas Martin Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care npj Digital Medicine |
| title | Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care |
| title_full | Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care |
| title_fullStr | Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care |
| title_full_unstemmed | Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care |
| title_short | Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care |
| title_sort | development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care |
| url | https://doi.org/10.1038/s41746-021-00388-6 |
| work_keys_str_mv | AT arnepeine developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT ahmedhallawa developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT johannesbickenbach developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT guidodartmann developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT lejlabegicfazlic developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT ankeschmeink developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT gerdascheid developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT christophthiemermann developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT andreasschuppert developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT ryankindle developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT leoceli developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT gernotmarx developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare AT lukasmartin developmentandvalidationofareinforcementlearningalgorithmtodynamicallyoptimizemechanicalventilationincriticalcare |