Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing
Abstract Background A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model t...
Main Authors: | , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMC
2021-07-01
|
Series: | BMC Medical Informatics and Decision Making |
Subjects: | |
Online Access: | https://doi.org/10.1186/s12911-021-01574-y |
_version_ | 1819153255592099840 |
---|---|
author | Yeshwant Reddy Chillakuru Shourya Munjal Benjamin Laguna Timothy L. Chen Gunvant R. Chaudhari Thienkhai Vu Youngho Seo Jared Narvid Jae Ho Sohn |
author_facet | Yeshwant Reddy Chillakuru Shourya Munjal Benjamin Laguna Timothy L. Chen Gunvant R. Chaudhari Thienkhai Vu Youngho Seo Jared Narvid Jae Ho Sohn |
author_sort | Yeshwant Reddy Chillakuru |
collection | DOAJ |
description | Abstract Background A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text. Methods We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku. Results The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment. Conclusion We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency. |
first_indexed | 2024-12-22T15:02:17Z |
format | Article |
id | doaj.art-f0657fccdd3747b78beddbed5ad57d3a |
institution | Directory Open Access Journal |
issn | 1472-6947 |
language | English |
last_indexed | 2024-12-22T15:02:17Z |
publishDate | 2021-07-01 |
publisher | BMC |
record_format | Article |
series | BMC Medical Informatics and Decision Making |
spelling | doaj.art-f0657fccdd3747b78beddbed5ad57d3a2022-12-21T18:22:06ZengBMCBMC Medical Informatics and Decision Making1472-69472021-07-0121111010.1186/s12911-021-01574-yDevelopment and web deployment of an automated neuroradiology MRI protocoling tool with natural language processingYeshwant Reddy Chillakuru0Shourya Munjal1Benjamin Laguna2Timothy L. Chen3Gunvant R. Chaudhari4Thienkhai Vu5Youngho Seo6Jared Narvid7Jae Ho Sohn8Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Radiology & Biomedical Imaging, University of California San Francisco (UCSF)Abstract Background A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text. Methods We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku. Results The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment. Conclusion We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.https://doi.org/10.1186/s12911-021-01574-yNatural language processingProtocolAutomationNeuroimaging |
spellingShingle | Yeshwant Reddy Chillakuru Shourya Munjal Benjamin Laguna Timothy L. Chen Gunvant R. Chaudhari Thienkhai Vu Youngho Seo Jared Narvid Jae Ho Sohn Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing BMC Medical Informatics and Decision Making Natural language processing Protocol Automation Neuroimaging |
title | Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing |
title_full | Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing |
title_fullStr | Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing |
title_full_unstemmed | Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing |
title_short | Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing |
title_sort | development and web deployment of an automated neuroradiology mri protocoling tool with natural language processing |
topic | Natural language processing Protocol Automation Neuroimaging |
url | https://doi.org/10.1186/s12911-021-01574-y |
work_keys_str_mv | AT yeshwantreddychillakuru developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT shouryamunjal developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT benjaminlaguna developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT timothylchen developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT gunvantrchaudhari developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT thienkhaivu developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT younghoseo developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT jarednarvid developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing AT jaehosohn developmentandwebdeploymentofanautomatedneuroradiologymriprotocolingtoolwithnaturallanguageprocessing |