Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study
BackgroundChronic obstructive pulmonary disease (COPD) is a major cause of death and places a heavy burden on health care. To optimize the allocation of precious preventive care management resources and improve the outcomes for high-risk patients with COPD, we recently built...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2022-02-01
|
| Series: | JMIR Medical Informatics |
| Online Access: | https://medinform.jmir.org/2022/2/e33043 |
| _version_ | 1797735244026085376 |
|---|---|
| author | Siyang Zeng Mehrdad Arjomandi Gang Luo |
| author_facet | Siyang Zeng Mehrdad Arjomandi Gang Luo |
| author_sort | Siyang Zeng |
| collection | DOAJ |
| description |
BackgroundChronic obstructive pulmonary disease (COPD) is a major cause of death and places a heavy burden on health care. To optimize the allocation of precious preventive care management resources and improve the outcomes for high-risk patients with COPD, we recently built the most accurate model to date to predict severe COPD exacerbations, which need inpatient stays or emergency department visits, in the following 12 months. Our model is a machine learning model. As is the case with most machine learning models, our model does not explain its predictions, forming a barrier for clinical use. Previously, we designed a method to automatically provide rule-type explanations for machine learning predictions and suggest tailored interventions with no loss of model performance. This method has been tested before for asthma outcome prediction but not for COPD outcome prediction.
ObjectiveThis study aims to assess the generalizability of our automatic explanation method for predicting severe COPD exacerbations.
MethodsThe patient cohort included all patients with COPD who visited the University of Washington Medicine facilities between 2011 and 2019. In a secondary analysis of 43,576 data instances, we used our formerly developed automatic explanation method to automatically explain our model’s predictions and suggest tailored interventions.
ResultsOur method explained the predictions for 97.1% (100/103) of the patients with COPD whom our model correctly predicted to have severe COPD exacerbations in the following 12 months and the predictions for 73.6% (134/182) of the patients with COPD who had ≥1 severe COPD exacerbation in the following 12 months.
ConclusionsOur automatic explanation method worked well for predicting severe COPD exacerbations. After further improving our method, we hope to use it to facilitate future clinical use of our model.
International Registered Report Identifier (IRRID)RR2-10.2196/13783 |
| first_indexed | 2024-03-12T12:56:11Z |
| format | Article |
| id | doaj.art-1d0415ba37794bc3ab178ecd3a9da178 |
| institution | Directory Open Access Journal |
| issn | 2291-9694 |
| language | English |
| last_indexed | 2024-03-12T12:56:11Z |
| publishDate | 2022-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Informatics |
| spelling | doaj.art-1d0415ba37794bc3ab178ecd3a9da1782023-08-28T20:53:19ZengJMIR PublicationsJMIR Medical Informatics2291-96942022-02-01102e3304310.2196/33043Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort StudySiyang Zenghttps://orcid.org/0000-0001-9346-301XMehrdad Arjomandihttps://orcid.org/0000-0002-0116-9217Gang Luohttps://orcid.org/0000-0001-7217-4008 BackgroundChronic obstructive pulmonary disease (COPD) is a major cause of death and places a heavy burden on health care. To optimize the allocation of precious preventive care management resources and improve the outcomes for high-risk patients with COPD, we recently built the most accurate model to date to predict severe COPD exacerbations, which need inpatient stays or emergency department visits, in the following 12 months. Our model is a machine learning model. As is the case with most machine learning models, our model does not explain its predictions, forming a barrier for clinical use. Previously, we designed a method to automatically provide rule-type explanations for machine learning predictions and suggest tailored interventions with no loss of model performance. This method has been tested before for asthma outcome prediction but not for COPD outcome prediction. ObjectiveThis study aims to assess the generalizability of our automatic explanation method for predicting severe COPD exacerbations. MethodsThe patient cohort included all patients with COPD who visited the University of Washington Medicine facilities between 2011 and 2019. In a secondary analysis of 43,576 data instances, we used our formerly developed automatic explanation method to automatically explain our model’s predictions and suggest tailored interventions. ResultsOur method explained the predictions for 97.1% (100/103) of the patients with COPD whom our model correctly predicted to have severe COPD exacerbations in the following 12 months and the predictions for 73.6% (134/182) of the patients with COPD who had ≥1 severe COPD exacerbation in the following 12 months. ConclusionsOur automatic explanation method worked well for predicting severe COPD exacerbations. After further improving our method, we hope to use it to facilitate future clinical use of our model. International Registered Report Identifier (IRRID)RR2-10.2196/13783https://medinform.jmir.org/2022/2/e33043 |
| spellingShingle | Siyang Zeng Mehrdad Arjomandi Gang Luo Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study JMIR Medical Informatics |
| title | Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study |
| title_full | Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study |
| title_fullStr | Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study |
| title_full_unstemmed | Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study |
| title_short | Automatically Explaining Machine Learning Predictions on Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study |
| title_sort | automatically explaining machine learning predictions on severe chronic obstructive pulmonary disease exacerbations retrospective cohort study |
| url | https://medinform.jmir.org/2022/2/e33043 |
| work_keys_str_mv | AT siyangzeng automaticallyexplainingmachinelearningpredictionsonseverechronicobstructivepulmonarydiseaseexacerbationsretrospectivecohortstudy AT mehrdadarjomandi automaticallyexplainingmachinelearningpredictionsonseverechronicobstructivepulmonarydiseaseexacerbationsretrospectivecohortstudy AT gangluo automaticallyexplainingmachinelearningpredictionsonseverechronicobstructivepulmonarydiseaseexacerbationsretrospectivecohortstudy |