Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture
Background: Machine learning (ML) has demonstrated success in classifying patients’ diagnostic outcomes in free-text clinical notes. However, due to the machine learning model's complexity, interpreting the mechanism behind classification results remains difficult. Methods: We investigated inte...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-01-01
|
| Series: | Computer Methods and Programs in Biomedicine Update |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666990023000137 |
| _version_ | 1827924326063538176 |
|---|---|
| author | Tong Ling Luo Jake Jazzmyne Adams Kristen Osinski Xiaoyu Liu David Friedland |
| author_facet | Tong Ling Luo Jake Jazzmyne Adams Kristen Osinski Xiaoyu Liu David Friedland |
| author_sort | Tong Ling |
| collection | DOAJ |
| description | Background: Machine learning (ML) has demonstrated success in classifying patients’ diagnostic outcomes in free-text clinical notes. However, due to the machine learning model's complexity, interpreting the mechanism behind classification results remains difficult. Methods: We investigated interpretable representations of text-based machine learning classification models. We created machine learning models to classify temporal bone fractures based on 164 temporal bone computed tomography (CT) text reports. We adopted the XGBoost, Support Vector Machine, Logistic Regression, and Random Forest algorithms. To interpret models, we used two major methodologies: (1) We calculated the average word frequency score (WFS) for keywords. The word frequency score shows the frequency gap between positively and negatively classified cases. (2) We used Local Interpretable Model-Agnostic Explanations (LIME) to show the word-level contribution to bone fracture classification. Results: In temporal bone fracture classification, the random forest model achieved an average F1-score of 0.93. WFS revealed a difference in keyword usage between fracture and non-fracture cases. Additionally, LIME visualized the keywords' contributions to the classification results. The evaluation of LIME-based interpretation achieved the highest interpreting accuracy of 0.97. Conclusion: The interpretable text explainer can improve physicians' understanding of machine learning predictions. By providing simple visualization, our model can increase the trust of computerized models. Our model supports more transparent computerized decision-making in clinical settings. |
| first_indexed | 2024-03-13T05:10:16Z |
| format | Article |
| id | doaj.art-bd4531bdd84d40b6915cf9d55730bd2d |
| institution | Directory Open Access Journal |
| issn | 2666-9900 |
| language | English |
| last_indexed | 2024-03-13T05:10:16Z |
| publishDate | 2023-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Computer Methods and Programs in Biomedicine Update |
| spelling | doaj.art-bd4531bdd84d40b6915cf9d55730bd2d2023-06-16T05:12:14ZengElsevierComputer Methods and Programs in Biomedicine Update2666-99002023-01-013100104Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fractureTong Ling0Luo Jake1Jazzmyne Adams2Kristen Osinski3Xiaoyu Liu4David Friedland5Department of Health Informatics and Administration, University of Wisconsin-Milwaukee, Milwaukee, USADepartment of Health Informatics and Administration, University of Wisconsin-Milwaukee, Milwaukee, USA; Corresponding author.Department of Otolaryngology and Communication Sciences, Medical College of Wisconsin, Milwaukee, WI, USAMedical College of Wisconsin, Clinical and Translational Science Institute of Southeastern Wisconsin, USADepartment of Electrical Engineering and Computer Science, University of Wisconsin–Milwaukee, Milwaukee, WI, USAMedical College of Wisconsin, Clinical and Translational Science Institute of Southeastern Wisconsin, USABackground: Machine learning (ML) has demonstrated success in classifying patients’ diagnostic outcomes in free-text clinical notes. However, due to the machine learning model's complexity, interpreting the mechanism behind classification results remains difficult. Methods: We investigated interpretable representations of text-based machine learning classification models. We created machine learning models to classify temporal bone fractures based on 164 temporal bone computed tomography (CT) text reports. We adopted the XGBoost, Support Vector Machine, Logistic Regression, and Random Forest algorithms. To interpret models, we used two major methodologies: (1) We calculated the average word frequency score (WFS) for keywords. The word frequency score shows the frequency gap between positively and negatively classified cases. (2) We used Local Interpretable Model-Agnostic Explanations (LIME) to show the word-level contribution to bone fracture classification. Results: In temporal bone fracture classification, the random forest model achieved an average F1-score of 0.93. WFS revealed a difference in keyword usage between fracture and non-fracture cases. Additionally, LIME visualized the keywords' contributions to the classification results. The evaluation of LIME-based interpretation achieved the highest interpreting accuracy of 0.97. Conclusion: The interpretable text explainer can improve physicians' understanding of machine learning predictions. By providing simple visualization, our model can increase the trust of computerized models. Our model supports more transparent computerized decision-making in clinical settings.http://www.sciencedirect.com/science/article/pii/S2666990023000137Interpretable machine learningArtificial intelligenceText classificationBone fractureComputed tomography |
| spellingShingle | Tong Ling Luo Jake Jazzmyne Adams Kristen Osinski Xiaoyu Liu David Friedland Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture Computer Methods and Programs in Biomedicine Update Interpretable machine learning Artificial intelligence Text classification Bone fracture Computed tomography |
| title | Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture |
| title_full | Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture |
| title_fullStr | Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture |
| title_full_unstemmed | Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture |
| title_short | Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture |
| title_sort | interpretable machine learning text classification for clinical computed tomography reports a case study of temporal bone fracture |
| topic | Interpretable machine learning Artificial intelligence Text classification Bone fracture Computed tomography |
| url | http://www.sciencedirect.com/science/article/pii/S2666990023000137 |
| work_keys_str_mv | AT tongling interpretablemachinelearningtextclassificationforclinicalcomputedtomographyreportsacasestudyoftemporalbonefracture AT luojake interpretablemachinelearningtextclassificationforclinicalcomputedtomographyreportsacasestudyoftemporalbonefracture AT jazzmyneadams interpretablemachinelearningtextclassificationforclinicalcomputedtomographyreportsacasestudyoftemporalbonefracture AT kristenosinski interpretablemachinelearningtextclassificationforclinicalcomputedtomographyreportsacasestudyoftemporalbonefracture AT xiaoyuliu interpretablemachinelearningtextclassificationforclinicalcomputedtomographyreportsacasestudyoftemporalbonefracture AT davidfriedland interpretablemachinelearningtextclassificationforclinicalcomputedtomographyreportsacasestudyoftemporalbonefracture |