Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach
Background The medical subdomain of a clinical note, such as cardiology or neurology, is useful content-derived metadata for developing machine learning downstream applications. To classify the medical subdomain of a note accurately, we have constructed a machine learning-based natu...
| Main Authors: | , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
BioMed Central
2018
|
| Online Access: | http://hdl.handle.net/1721.1/114596 https://orcid.org/0000-0003-2232-0390 https://orcid.org/0000-0001-8411-6403 |
| _version_ | 1826191353177964544 |
|---|---|
| author | Wagholikar, Kavishwar B McCray, Alexa T Chueh, Henry C Wagholikar, Kavishwar B. McCray, Alexa T. Chueh, Henry C. Weng, Wei-Hung Szolovits, Peter |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Wagholikar, Kavishwar B McCray, Alexa T Chueh, Henry C Wagholikar, Kavishwar B. McCray, Alexa T. Chueh, Henry C. Weng, Wei-Hung Szolovits, Peter |
| author_sort | Wagholikar, Kavishwar B |
| collection | MIT |
| description | Background
The medical subdomain of a clinical note, such as cardiology or neurology, is useful content-derived metadata for developing machine learning downstream applications. To classify the medical subdomain of a note accurately, we have constructed a machine learning-based natural language processing (NLP) pipeline and developed medical subdomain classifiers based on the content of the note.
Methods
We constructed the pipeline using the clinical NLP system, clinical Text Analysis and Knowledge Extraction System (cTAKES), the Unified Medical Language System (UMLS) Metathesaurus, Semantic Network, and learning algorithms to extract features from two datasets — clinical notes from Integrating Data for Analysis, Anonymization, and Sharing (iDASH) data repository (n = 431) and Massachusetts General Hospital (MGH) (n = 91,237), and built medical subdomain classifiers with different combinations of data representation methods and supervised learning algorithms. We evaluated the performance of classifiers and their portability across the two datasets.
Results
The convolutional recurrent neural network with neural word embeddings trained-medical subdomain classifier yielded the best performance measurement on iDASH and MGH datasets with area under receiver operating characteristic curve (AUC) of 0.975 and 0.991, and F1 scores of 0.845 and 0.870, respectively. Considering better clinical interpretability, linear support vector machine-trained medical subdomain classifier using hybrid bag-of-words and clinically relevant UMLS concepts as the feature representation, with term frequency-inverse document frequency (tf-idf)-weighting, outperformed other shallow learning classifiers on iDASH and MGH datasets with AUC of 0.957 and 0.964, and F1 scores of 0.932 and 0.934 respectively. We trained classifiers on one dataset, applied to the other dataset and yielded the threshold of F1 score of 0.7 in classifiers for half of the medical subdomains we studied.
Conclusion
Our study shows that a supervised learning-based NLP approach is useful to develop medical subdomain classifiers. The deep learning algorithm with distributed word representation yields better performance yet shallow learning algorithms with the word and concept representation achieves comparable performance with better clinical interpretability. Portable classifiers may also be used across datasets from different institutions. Keywords: Medical Decision Making; Computer-assisted; Natural Language Processing; Unified Medical Language System; Machine Learning; Deep Learning; Distributed Representation |
| first_indexed | 2024-09-23T08:54:56Z |
| format | Article |
| id | mit-1721.1/114596 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T08:54:56Z |
| publishDate | 2018 |
| publisher | BioMed Central |
| record_format | dspace |
| spelling | mit-1721.1/1145962022-09-26T09:09:31Z Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach Wagholikar, Kavishwar B McCray, Alexa T Chueh, Henry C Wagholikar, Kavishwar B. McCray, Alexa T. Chueh, Henry C. Weng, Wei-Hung Szolovits, Peter Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Weng, Wei-Hung Szolovits, Peter Background The medical subdomain of a clinical note, such as cardiology or neurology, is useful content-derived metadata for developing machine learning downstream applications. To classify the medical subdomain of a note accurately, we have constructed a machine learning-based natural language processing (NLP) pipeline and developed medical subdomain classifiers based on the content of the note. Methods We constructed the pipeline using the clinical NLP system, clinical Text Analysis and Knowledge Extraction System (cTAKES), the Unified Medical Language System (UMLS) Metathesaurus, Semantic Network, and learning algorithms to extract features from two datasets — clinical notes from Integrating Data for Analysis, Anonymization, and Sharing (iDASH) data repository (n = 431) and Massachusetts General Hospital (MGH) (n = 91,237), and built medical subdomain classifiers with different combinations of data representation methods and supervised learning algorithms. We evaluated the performance of classifiers and their portability across the two datasets. Results The convolutional recurrent neural network with neural word embeddings trained-medical subdomain classifier yielded the best performance measurement on iDASH and MGH datasets with area under receiver operating characteristic curve (AUC) of 0.975 and 0.991, and F1 scores of 0.845 and 0.870, respectively. Considering better clinical interpretability, linear support vector machine-trained medical subdomain classifier using hybrid bag-of-words and clinically relevant UMLS concepts as the feature representation, with term frequency-inverse document frequency (tf-idf)-weighting, outperformed other shallow learning classifiers on iDASH and MGH datasets with AUC of 0.957 and 0.964, and F1 scores of 0.932 and 0.934 respectively. We trained classifiers on one dataset, applied to the other dataset and yielded the threshold of F1 score of 0.7 in classifiers for half of the medical subdomains we studied. Conclusion Our study shows that a supervised learning-based NLP approach is useful to develop medical subdomain classifiers. The deep learning algorithm with distributed word representation yields better performance yet shallow learning algorithms with the word and concept representation achieves comparable performance with better clinical interpretability. Portable classifiers may also be used across datasets from different institutions. Keywords: Medical Decision Making; Computer-assisted; Natural Language Processing; Unified Medical Language System; Machine Learning; Deep Learning; Distributed Representation 2018-04-06T16:53:28Z 2018-04-06T16:53:28Z 2017-12 2017-06 2017-12-03T04:22:13Z Article http://purl.org/eprint/type/JournalArticle 1472-6947 http://hdl.handle.net/1721.1/114596 Weng, Wei-Hung et al. "Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach." BMC Medical Informatics and Decision Making 17 (December 2017): 155 © 2017 The Author(s) https://orcid.org/0000-0003-2232-0390 https://orcid.org/0000-0001-8411-6403 en http://dx.doi.org/10.1186/s12911-017-0556-8 BMC Medical Informatics and Decision Making Creative Commons Attribution http://creativecommons.org/licenses/by/4.0/ The Author(s). application/pdf BioMed Central BioMed Central |
| spellingShingle | Wagholikar, Kavishwar B McCray, Alexa T Chueh, Henry C Wagholikar, Kavishwar B. McCray, Alexa T. Chueh, Henry C. Weng, Wei-Hung Szolovits, Peter Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach |
| title | Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach |
| title_full | Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach |
| title_fullStr | Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach |
| title_full_unstemmed | Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach |
| title_short | Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach |
| title_sort | medical subdomain classification of clinical notes using a machine learning based natural language processing approach |
| url | http://hdl.handle.net/1721.1/114596 https://orcid.org/0000-0003-2232-0390 https://orcid.org/0000-0001-8411-6403 |
| work_keys_str_mv | AT wagholikarkavishwarb medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT mccrayalexat medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT chuehhenryc medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT wagholikarkavishwarb medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT mccrayalexat medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT chuehhenryc medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT wengweihung medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach AT szolovitspeter medicalsubdomainclassificationofclinicalnotesusingamachinelearningbasednaturallanguageprocessingapproach |