A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records
Abstract Background The Named Entity Recognition (NER) task as a key step in the extraction of health information, has encountered many challenges in Chinese Electronic Medical Records (EMRs). Firstly, the casual use of Chinese abbreviations and doctors’ personal style may result in multiple express...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2019-04-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12911-019-0762-7 |
| _version_ | 1818923889536794624 |
|---|---|
| author | Xiaoling Cai Shoubin Dong Jinlong Hu |
| author_facet | Xiaoling Cai Shoubin Dong Jinlong Hu |
| author_sort | Xiaoling Cai |
| collection | DOAJ |
| description | Abstract Background The Named Entity Recognition (NER) task as a key step in the extraction of health information, has encountered many challenges in Chinese Electronic Medical Records (EMRs). Firstly, the casual use of Chinese abbreviations and doctors’ personal style may result in multiple expressions of the same entity, and we lack a common Chinese medical dictionary to perform accurate entity extraction. Secondly, the electronic medical record contains entities from a variety of categories of entities, and the length of those entities in different categories varies greatly, which increases the difficult in the extraction for the Chinese NER. Therefore, the entity boundary detection becomes the key to perform accurate entity extraction of Chinese EMRs, and we need to develop a model that supports multiple length entity recognition without relying on any medical dictionary. Methods In this study, we incorporate part-of-speech (POS) information into the deep learning model to improve the accuracy of Chinese entity boundary detection. In order to avoid the wrongly POS tagging of long entities, we proposed a method called reduced POS tagging that reserves the tags of general words but not of the seemingly medical entities. The model proposed in this paper, named SM-LSTM-CRF, consists of three layers: self-matching attention layer – calculating the relevance of each character to the entire sentence; LSTM (Long Short-Term Memory) layer – capturing the context feature of each character; CRF (Conditional Random Field) layer – labeling characters based on their features and transfer rules. Results The experimental results at a Chinese EMRs dataset show that the F1 value of SM-LSTM-CRF is increased by 2.59% compared to that of the LSTM-CRF. After adding POS feature in the model, we get an improvement of about 7.74% at F1. The reduced POS tagging reduces the false tagging on long entities, thus increases the F1 value by 2.42% and achieves an F1 score of 80.07%. Conclusions The POS feature marked by the reduced POS tagging together with self-matching attention mechanism puts a stranglehold on entity boundaries and has a good performance in the recognition of clinical entities. |
| first_indexed | 2024-12-20T02:16:36Z |
| format | Article |
| id | doaj.art-c7920f89012240bdbebe3dd0d768b841 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-12-20T02:16:36Z |
| publishDate | 2019-04-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-c7920f89012240bdbebe3dd0d768b8412022-12-21T19:56:55ZengBMCBMC Medical Informatics and Decision Making1472-69472019-04-0119S210110910.1186/s12911-019-0762-7A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical recordsXiaoling Cai0Shoubin Dong1Jinlong Hu2Communication & Computer Network Lab of Guangdong, School of Computer Science and Engineering, South China University of TechnologyCommunication & Computer Network Lab of Guangdong, School of Computer Science and Engineering, South China University of TechnologyCommunication & Computer Network Lab of Guangdong, School of Computer Science and Engineering, South China University of TechnologyAbstract Background The Named Entity Recognition (NER) task as a key step in the extraction of health information, has encountered many challenges in Chinese Electronic Medical Records (EMRs). Firstly, the casual use of Chinese abbreviations and doctors’ personal style may result in multiple expressions of the same entity, and we lack a common Chinese medical dictionary to perform accurate entity extraction. Secondly, the electronic medical record contains entities from a variety of categories of entities, and the length of those entities in different categories varies greatly, which increases the difficult in the extraction for the Chinese NER. Therefore, the entity boundary detection becomes the key to perform accurate entity extraction of Chinese EMRs, and we need to develop a model that supports multiple length entity recognition without relying on any medical dictionary. Methods In this study, we incorporate part-of-speech (POS) information into the deep learning model to improve the accuracy of Chinese entity boundary detection. In order to avoid the wrongly POS tagging of long entities, we proposed a method called reduced POS tagging that reserves the tags of general words but not of the seemingly medical entities. The model proposed in this paper, named SM-LSTM-CRF, consists of three layers: self-matching attention layer – calculating the relevance of each character to the entire sentence; LSTM (Long Short-Term Memory) layer – capturing the context feature of each character; CRF (Conditional Random Field) layer – labeling characters based on their features and transfer rules. Results The experimental results at a Chinese EMRs dataset show that the F1 value of SM-LSTM-CRF is increased by 2.59% compared to that of the LSTM-CRF. After adding POS feature in the model, we get an improvement of about 7.74% at F1. The reduced POS tagging reduces the false tagging on long entities, thus increases the F1 value by 2.42% and achieves an F1 score of 80.07%. Conclusions The POS feature marked by the reduced POS tagging together with self-matching attention mechanism puts a stranglehold on entity boundaries and has a good performance in the recognition of clinical entities.http://link.springer.com/article/10.1186/s12911-019-0762-7Part of speechChinese electronic medical recordsNamed entity recognition |
| spellingShingle | Xiaoling Cai Shoubin Dong Jinlong Hu A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records BMC Medical Informatics and Decision Making Part of speech Chinese electronic medical records Named entity recognition |
| title | A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records |
| title_full | A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records |
| title_fullStr | A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records |
| title_full_unstemmed | A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records |
| title_short | A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records |
| title_sort | deep learning model incorporating part of speech and self matching attention for named entity recognition of chinese electronic medical records |
| topic | Part of speech Chinese electronic medical records Named entity recognition |
| url | http://link.springer.com/article/10.1186/s12911-019-0762-7 |
| work_keys_str_mv | AT xiaolingcai adeeplearningmodelincorporatingpartofspeechandselfmatchingattentionfornamedentityrecognitionofchineseelectronicmedicalrecords AT shoubindong adeeplearningmodelincorporatingpartofspeechandselfmatchingattentionfornamedentityrecognitionofchineseelectronicmedicalrecords AT jinlonghu adeeplearningmodelincorporatingpartofspeechandselfmatchingattentionfornamedentityrecognitionofchineseelectronicmedicalrecords AT xiaolingcai deeplearningmodelincorporatingpartofspeechandselfmatchingattentionfornamedentityrecognitionofchineseelectronicmedicalrecords AT shoubindong deeplearningmodelincorporatingpartofspeechandselfmatchingattentionfornamedentityrecognitionofchineseelectronicmedicalrecords AT jinlonghu deeplearningmodelincorporatingpartofspeechandselfmatchingattentionfornamedentityrecognitionofchineseelectronicmedicalrecords |