Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients
BackgroundElectronic health record (EHR) systems contain a large volume of texts, including visit notes, discharge summaries, and various reports. To protect the confidentiality of patients, these records often need to be fully de-identified before circulating for secondary use. Machine learning (ML...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-02-01
|
| Series: | Frontiers in Digital Health |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fdgth.2022.728922/full |
| _version_ | 1819281304856821760 |
|---|---|
| author | Tanmoy Paul Tanmoy Paul Md Kamruz Zaman Rana Md Kamruz Zaman Rana Preethi Aishwarya Tautam Teja Venkat Pavan Kotapati Yaswitha Jampani Yaswitha Jampani Nitesh Singh Nitesh Singh Humayera Islam Humayera Islam Vasanthi Mandhadi Vasanthi Mandhadi Vishakha Sharma Michael Barnes Richard D. Hammer Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa |
| author_facet | Tanmoy Paul Tanmoy Paul Md Kamruz Zaman Rana Md Kamruz Zaman Rana Preethi Aishwarya Tautam Teja Venkat Pavan Kotapati Yaswitha Jampani Yaswitha Jampani Nitesh Singh Nitesh Singh Humayera Islam Humayera Islam Vasanthi Mandhadi Vasanthi Mandhadi Vishakha Sharma Michael Barnes Richard D. Hammer Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa |
| author_sort | Tanmoy Paul |
| collection | DOAJ |
| description | BackgroundElectronic health record (EHR) systems contain a large volume of texts, including visit notes, discharge summaries, and various reports. To protect the confidentiality of patients, these records often need to be fully de-identified before circulating for secondary use. Machine learning (ML) based named entity recognition (NER) model has emerged as a popular technique of automatic de-identification.ObjectiveThe performance of a machine learning model highly depends on the selection of appropriate features. The objective of this study was to investigate the usability of multiple features in building a conditional random field (CRF) based clinical de-identification NER model.MethodsUsing open-source natural language processing (NLP) toolkits, we annotated protected health information (PHI) in 1,500 pathology reports and built supervised NER models using multiple features and their combinations. We further investigated the dependency of a model's performance on the size of training data.ResultsAmong the 10 feature extractors explored in this study, n-gram, prefix–suffix, word embedding, and word shape performed the best. A model using combination of these four feature sets yielded precision, recall, and F1-score for each PHI as follows: NAME (0.80; 0.79; 0.80), LOCATION (0.85; 0.83; 0.84), DATE (0.86; 0.79; 0.82), HOSPITAL (0.96; 0.93; 0.95), ID (0.99; 0.82; 0.90), and INITIALS (0.97; 0.49; 0.65). We also found that the model's performance becomes saturated when the training data size is beyond 200.ConclusionManual de-identification of large-scale data is an impractical procedure since it is time-consuming and subject to human errors. Analysis of the NER model's performance in this study sheds light on a semi-automatic clinical de-identification pipeline for enterprise-wide data warehousing. |
| first_indexed | 2024-12-24T00:57:34Z |
| format | Article |
| id | doaj.art-3828d9965431427887fb90453f48bc56 |
| institution | Directory Open Access Journal |
| issn | 2673-253X |
| language | English |
| last_indexed | 2024-12-24T00:57:34Z |
| publishDate | 2022-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Digital Health |
| spelling | doaj.art-3828d9965431427887fb90453f48bc562022-12-21T17:23:26ZengFrontiers Media S.A.Frontiers in Digital Health2673-253X2022-02-01410.3389/fdgth.2022.728922728922Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC PatientsTanmoy Paul0Tanmoy Paul1Md Kamruz Zaman Rana2Md Kamruz Zaman Rana3Preethi Aishwarya Tautam4Teja Venkat Pavan Kotapati5Yaswitha Jampani6Yaswitha Jampani7Nitesh Singh8Nitesh Singh9Humayera Islam10Humayera Islam11Vasanthi Mandhadi12Vasanthi Mandhadi13Vishakha Sharma14Michael Barnes15Richard D. Hammer16Abu Saleh Mohammad Mosa17Abu Saleh Mohammad Mosa18Abu Saleh Mohammad Mosa19Abu Saleh Mohammad Mosa20Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesDepartment of Health Management and Informatics, School of Medicine, University of Missouri, Columbia, MO, United StatesDepartment of Health Management and Informatics, School of Medicine, University of Missouri, Columbia, MO, United StatesDepartment of Health Management and Informatics, School of Medicine, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesDepartment of Health Management and Informatics, School of Medicine, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesDepartment of Health Management and Informatics, School of Medicine, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesInstitute for Data Science and Informatics, University of Missouri, Columbia, MO, United StatesDepartment of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesRoche Diagnostics, F. Hoffmann-La Roche, Santa Clara, CA, United StatesRoche Diagnostics, F. Hoffmann-La Roche, Santa Clara, CA, United StatesDepartment of Pathology and Anatomical Sciences, University of Missouri, Columbia, MO, United StatesDepartment of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, United StatesCenter for Biomedical Informatics, University of Missouri, Columbia, MO, United StatesDepartment of Health Management and Informatics, School of Medicine, University of Missouri, Columbia, MO, United StatesInstitute for Data Science and Informatics, University of Missouri, Columbia, MO, United StatesBackgroundElectronic health record (EHR) systems contain a large volume of texts, including visit notes, discharge summaries, and various reports. To protect the confidentiality of patients, these records often need to be fully de-identified before circulating for secondary use. Machine learning (ML) based named entity recognition (NER) model has emerged as a popular technique of automatic de-identification.ObjectiveThe performance of a machine learning model highly depends on the selection of appropriate features. The objective of this study was to investigate the usability of multiple features in building a conditional random field (CRF) based clinical de-identification NER model.MethodsUsing open-source natural language processing (NLP) toolkits, we annotated protected health information (PHI) in 1,500 pathology reports and built supervised NER models using multiple features and their combinations. We further investigated the dependency of a model's performance on the size of training data.ResultsAmong the 10 feature extractors explored in this study, n-gram, prefix–suffix, word embedding, and word shape performed the best. A model using combination of these four feature sets yielded precision, recall, and F1-score for each PHI as follows: NAME (0.80; 0.79; 0.80), LOCATION (0.85; 0.83; 0.84), DATE (0.86; 0.79; 0.82), HOSPITAL (0.96; 0.93; 0.95), ID (0.99; 0.82; 0.90), and INITIALS (0.97; 0.49; 0.65). We also found that the model's performance becomes saturated when the training data size is beyond 200.ConclusionManual de-identification of large-scale data is an impractical procedure since it is time-consuming and subject to human errors. Analysis of the NER model's performance in this study sheds light on a semi-automatic clinical de-identification pipeline for enterprise-wide data warehousing.https://www.frontiersin.org/articles/10.3389/fdgth.2022.728922/fullclinical text de-identificationprotected health informationNLPnamed entity recognitionde-identificationconditional random field |
| spellingShingle | Tanmoy Paul Tanmoy Paul Md Kamruz Zaman Rana Md Kamruz Zaman Rana Preethi Aishwarya Tautam Teja Venkat Pavan Kotapati Yaswitha Jampani Yaswitha Jampani Nitesh Singh Nitesh Singh Humayera Islam Humayera Islam Vasanthi Mandhadi Vasanthi Mandhadi Vishakha Sharma Michael Barnes Richard D. Hammer Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Abu Saleh Mohammad Mosa Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients Frontiers in Digital Health clinical text de-identification protected health information NLP named entity recognition de-identification conditional random field |
| title | Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients |
| title_full | Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients |
| title_fullStr | Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients |
| title_full_unstemmed | Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients |
| title_short | Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients |
| title_sort | investigation of the utility of features in a clinical de identification model a demonstration using ehr pathology reports for advanced nsclc patients |
| topic | clinical text de-identification protected health information NLP named entity recognition de-identification conditional random field |
| url | https://www.frontiersin.org/articles/10.3389/fdgth.2022.728922/full |
| work_keys_str_mv | AT tanmoypaul investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT tanmoypaul investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT mdkamruzzamanrana investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT mdkamruzzamanrana investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT preethiaishwaryatautam investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT tejavenkatpavankotapati investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT yaswithajampani investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT yaswithajampani investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT niteshsingh investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT niteshsingh investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT humayeraislam investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT humayeraislam investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT vasanthimandhadi investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT vasanthimandhadi investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT vishakhasharma investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT michaelbarnes investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT richarddhammer investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT abusalehmohammadmosa investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT abusalehmohammadmosa investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT abusalehmohammadmosa investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients AT abusalehmohammadmosa investigationoftheutilityoffeaturesinaclinicaldeidentificationmodelademonstrationusingehrpathologyreportsforadvancednsclcpatients |