Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study
BackgroundAcute neurological injury is a leading cause of permanent disability and death in the pediatric intensive care unit (PICU). No predictive model has been validated for critically ill children with acute neurological injury.ObjectivesWe hypothesized that PICU patients with concern for acute...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-06-01
|
| Series: | Frontiers in Pediatrics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fped.2023.1177470/full |
| _version_ | 1797793723078148096 |
|---|---|
| author | Neil K. Munjal Neil K. Munjal Robert S. B. Clark Dennis W. Simon Patrick M. Kochanek Christopher M. Horvat |
| author_facet | Neil K. Munjal Neil K. Munjal Robert S. B. Clark Dennis W. Simon Patrick M. Kochanek Christopher M. Horvat |
| author_sort | Neil K. Munjal |
| collection | DOAJ |
| description | BackgroundAcute neurological injury is a leading cause of permanent disability and death in the pediatric intensive care unit (PICU). No predictive model has been validated for critically ill children with acute neurological injury.ObjectivesWe hypothesized that PICU patients with concern for acute neurological injury are at higher risk for morbidity and mortality, and advanced analytics would derive robust, explainable subgroup models.MethodsWe performed a secondary subgroup analysis of the Trichotomous Outcomes in Pediatric Critical Care (TOPICC) study (2011–2013), predicting mortality and morbidity from admission physiology (lab values and vital signs in 6 h surrounding admission). We analyzed patients with suspected acute neurological injury using standard machine learning algorithms. Feature importance was analyzed using SHapley Additive exPlanations (SHAP). We created a Fast Healthcare Interoperability Resources (FHIR) application to demonstrate potential for interoperability using pragmatic data.Results1,860 patients had suspected acute neurological injury at PICU admission, with higher morbidity (8.2 vs. 3.4%) and mortality (6.2 vs. 1.9%) than those without similar concern. The ensemble regressor (containing Random Forest, Gradient Boosting, and Support Vector Machine learners) produced the best model, with Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.91 [95% CI (0.88, 0.94)] and Average Precision (AP) of 0.59 [0.51, 0.69] for mortality, and decreased performance predicting simultaneous mortality and morbidity (0.83 [0.80, 0.86] and 0.59 [0.51, 0.64]); at a set specificity of 0.995, positive predictive value (PPV) was 0.79 for mortality, and 0.88 for mortality and morbidity. By comparison, for mortality, the TOPICC logistic regression had AUROC of 0.90 [0.84, 0.93], but substantially inferior AP of 0.49 [0.35, 0.56] and PPV of 0.60 at specificity 0.995. Feature importance analysis showed that pupillary non-reactivity, Glasgow Coma Scale, and temperature were the most contributory vital signs, and acidosis and coagulopathy the most important laboratory values. The FHIR application provided a simulated demonstration of real-time health record query and model deployment.ConclusionsPICU patients with suspected acute neurological injury have higher mortality and morbidity. Our machine learning approach independently identified previously-known causes of secondary brain injury. Advanced modeling achieves improved positive predictive value in this important population compared to published models, providing a stepping stone in the path to deploying explainable models as interoperable bedside decision-support tools. |
| first_indexed | 2024-03-13T02:52:26Z |
| format | Article |
| id | doaj.art-58bb97b6d4d445e790b37217efd39147 |
| institution | Directory Open Access Journal |
| issn | 2296-2360 |
| language | English |
| last_indexed | 2024-03-13T02:52:26Z |
| publishDate | 2023-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Pediatrics |
| spelling | doaj.art-58bb97b6d4d445e790b37217efd391472023-06-28T09:35:22ZengFrontiers Media S.A.Frontiers in Pediatrics2296-23602023-06-011110.3389/fped.2023.11774701177470Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC studyNeil K. Munjal0Neil K. Munjal1Robert S. B. Clark2Dennis W. Simon3Patrick M. Kochanek4Christopher M. Horvat5Department of Pediatrics, University of Wisconsin—Madison, Madison, WI, United StatesDepartment of Critical Care Medicine, UPMC Children’s Hospital of Pittsburgh, Pittsburgh, PA, United StatesDepartment of Critical Care Medicine, UPMC Children’s Hospital of Pittsburgh, Pittsburgh, PA, United StatesDepartment of Critical Care Medicine, UPMC Children’s Hospital of Pittsburgh, Pittsburgh, PA, United StatesSafar Center for Resuscitation Research, University of Pittsburgh, Pittsburgh, PA, United StatesDepartment of Critical Care Medicine, UPMC Children’s Hospital of Pittsburgh, Pittsburgh, PA, United StatesBackgroundAcute neurological injury is a leading cause of permanent disability and death in the pediatric intensive care unit (PICU). No predictive model has been validated for critically ill children with acute neurological injury.ObjectivesWe hypothesized that PICU patients with concern for acute neurological injury are at higher risk for morbidity and mortality, and advanced analytics would derive robust, explainable subgroup models.MethodsWe performed a secondary subgroup analysis of the Trichotomous Outcomes in Pediatric Critical Care (TOPICC) study (2011–2013), predicting mortality and morbidity from admission physiology (lab values and vital signs in 6 h surrounding admission). We analyzed patients with suspected acute neurological injury using standard machine learning algorithms. Feature importance was analyzed using SHapley Additive exPlanations (SHAP). We created a Fast Healthcare Interoperability Resources (FHIR) application to demonstrate potential for interoperability using pragmatic data.Results1,860 patients had suspected acute neurological injury at PICU admission, with higher morbidity (8.2 vs. 3.4%) and mortality (6.2 vs. 1.9%) than those without similar concern. The ensemble regressor (containing Random Forest, Gradient Boosting, and Support Vector Machine learners) produced the best model, with Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.91 [95% CI (0.88, 0.94)] and Average Precision (AP) of 0.59 [0.51, 0.69] for mortality, and decreased performance predicting simultaneous mortality and morbidity (0.83 [0.80, 0.86] and 0.59 [0.51, 0.64]); at a set specificity of 0.995, positive predictive value (PPV) was 0.79 for mortality, and 0.88 for mortality and morbidity. By comparison, for mortality, the TOPICC logistic regression had AUROC of 0.90 [0.84, 0.93], but substantially inferior AP of 0.49 [0.35, 0.56] and PPV of 0.60 at specificity 0.995. Feature importance analysis showed that pupillary non-reactivity, Glasgow Coma Scale, and temperature were the most contributory vital signs, and acidosis and coagulopathy the most important laboratory values. The FHIR application provided a simulated demonstration of real-time health record query and model deployment.ConclusionsPICU patients with suspected acute neurological injury have higher mortality and morbidity. Our machine learning approach independently identified previously-known causes of secondary brain injury. Advanced modeling achieves improved positive predictive value in this important population compared to published models, providing a stepping stone in the path to deploying explainable models as interoperable bedside decision-support tools.https://www.frontiersin.org/articles/10.3389/fped.2023.1177470/fullmachine learningclinical decision supportpediatric intensive care unitacute neurological injurypredictive modeling |
| spellingShingle | Neil K. Munjal Neil K. Munjal Robert S. B. Clark Dennis W. Simon Patrick M. Kochanek Christopher M. Horvat Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study Frontiers in Pediatrics machine learning clinical decision support pediatric intensive care unit acute neurological injury predictive modeling |
| title | Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study |
| title_full | Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study |
| title_fullStr | Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study |
| title_full_unstemmed | Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study |
| title_short | Interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit: secondary analysis of the TOPICC study |
| title_sort | interoperable and explainable machine learning models to predict morbidity and mortality in acute neurological injury in the pediatric intensive care unit secondary analysis of the topicc study |
| topic | machine learning clinical decision support pediatric intensive care unit acute neurological injury predictive modeling |
| url | https://www.frontiersin.org/articles/10.3389/fped.2023.1177470/full |
| work_keys_str_mv | AT neilkmunjal interoperableandexplainablemachinelearningmodelstopredictmorbidityandmortalityinacuteneurologicalinjuryinthepediatricintensivecareunitsecondaryanalysisofthetopiccstudy AT neilkmunjal interoperableandexplainablemachinelearningmodelstopredictmorbidityandmortalityinacuteneurologicalinjuryinthepediatricintensivecareunitsecondaryanalysisofthetopiccstudy AT robertsbclark interoperableandexplainablemachinelearningmodelstopredictmorbidityandmortalityinacuteneurologicalinjuryinthepediatricintensivecareunitsecondaryanalysisofthetopiccstudy AT denniswsimon interoperableandexplainablemachinelearningmodelstopredictmorbidityandmortalityinacuteneurologicalinjuryinthepediatricintensivecareunitsecondaryanalysisofthetopiccstudy AT patrickmkochanek interoperableandexplainablemachinelearningmodelstopredictmorbidityandmortalityinacuteneurologicalinjuryinthepediatricintensivecareunitsecondaryanalysisofthetopiccstudy AT christophermhorvat interoperableandexplainablemachinelearningmodelstopredictmorbidityandmortalityinacuteneurologicalinjuryinthepediatricintensivecareunitsecondaryanalysisofthetopiccstudy |