Automatic Incident Triage in Radiation Oncology Incident Learning System
The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-08-01
|
| Series: | Healthcare |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-9032/8/3/272 |
| _version_ | 1797558110729011200 |
|---|---|
| author | Khajamoinuddin Syed William Sleeman Michael Hagan Jatinder Palta Rishabh Kapoor Preetam Ghosh |
| author_facet | Khajamoinuddin Syed William Sleeman Michael Hagan Jatinder Palta Rishabh Kapoor Preetam Ghosh |
| author_sort | Khajamoinuddin Syed |
| collection | DOAJ |
| description | The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports. |
| first_indexed | 2024-03-10T17:25:49Z |
| format | Article |
| id | doaj.art-e184b980c4b141bfafd489e496531580 |
| institution | Directory Open Access Journal |
| issn | 2227-9032 |
| language | English |
| last_indexed | 2024-03-10T17:25:49Z |
| publishDate | 2020-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Healthcare |
| spelling | doaj.art-e184b980c4b141bfafd489e4965315802023-11-20T10:10:54ZengMDPI AGHealthcare2227-90322020-08-018327210.3390/healthcare8030272Automatic Incident Triage in Radiation Oncology Incident Learning SystemKhajamoinuddin Syed0William Sleeman1Michael Hagan2Jatinder Palta3Rishabh Kapoor4Preetam Ghosh5Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USADepartment of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USADepartment of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USADepartment of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USADepartment of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USADepartment of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USAThe Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports.https://www.mdpi.com/2227-9032/8/3/272incident learning systemdeep learningautomated triagenatural language processingtransfer learning |
| spellingShingle | Khajamoinuddin Syed William Sleeman Michael Hagan Jatinder Palta Rishabh Kapoor Preetam Ghosh Automatic Incident Triage in Radiation Oncology Incident Learning System Healthcare incident learning system deep learning automated triage natural language processing transfer learning |
| title | Automatic Incident Triage in Radiation Oncology Incident Learning System |
| title_full | Automatic Incident Triage in Radiation Oncology Incident Learning System |
| title_fullStr | Automatic Incident Triage in Radiation Oncology Incident Learning System |
| title_full_unstemmed | Automatic Incident Triage in Radiation Oncology Incident Learning System |
| title_short | Automatic Incident Triage in Radiation Oncology Incident Learning System |
| title_sort | automatic incident triage in radiation oncology incident learning system |
| topic | incident learning system deep learning automated triage natural language processing transfer learning |
| url | https://www.mdpi.com/2227-9032/8/3/272 |
| work_keys_str_mv | AT khajamoinuddinsyed automaticincidenttriageinradiationoncologyincidentlearningsystem AT williamsleeman automaticincidenttriageinradiationoncologyincidentlearningsystem AT michaelhagan automaticincidenttriageinradiationoncologyincidentlearningsystem AT jatinderpalta automaticincidenttriageinradiationoncologyincidentlearningsystem AT rishabhkapoor automaticincidenttriageinradiationoncologyincidentlearningsystem AT preetamghosh automaticincidenttriageinradiationoncologyincidentlearningsystem |