An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records
Abstract Background Clinical named entity recognition (CNER) is important for medical information mining and establishment of high-quality knowledge map. Due to the different text features from natural language and a large number of professional and uncommon clinical terms in Chinese electronic medi...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2019-12-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12911-019-0933-6 |
| _version_ | 1818352273495949312 |
|---|---|
| author | Luqi Li Jie Zhao Li Hou Yunkai Zhai Jinming Shi Fangfang Cui |
| author_facet | Luqi Li Jie Zhao Li Hou Yunkai Zhai Jinming Shi Fangfang Cui |
| author_sort | Luqi Li |
| collection | DOAJ |
| description | Abstract Background Clinical named entity recognition (CNER) is important for medical information mining and establishment of high-quality knowledge map. Due to the different text features from natural language and a large number of professional and uncommon clinical terms in Chinese electronic medical records (EMRs), there are still many difficulties in clinical named entity recognition of Chinese EMRs. It is of great importance to eliminate semantic interference and improve the ability of autonomous learning of internal features of the model under the small training corpus. Methods From the perspective of deep learning, we integrated the attention mechanism into neural network, and proposed an improved clinical named entity recognition method for Chinese electronic medical records called BiLSTM-Att-CRF, which could capture more useful information of the context and avoid the problem of missing information caused by long-distance factors. In addition, medical dictionaries and part-of-speech (POS) features were also introduced to improve the performance of the model. Results Based on China Conference on Knowledge Graph and Semantic Computing (CCKS) 2017 and 2018 Chinese EMRs corpus, our BiLSTM-Att-CRF model finally achieved better performance than other widely-used models without additional features(F1-measure of 85.4% in CCKS 2018, F1-measure of 90.29% in CCKS 2017), and achieved the best performance with POS and dictionary features (F1-measure of 86.11% in CCKS 2018, F1-measure of 90.48% in CCKS 2017). In particular, the BiLSTM-Att-CRF model had significant effect on the improvement of Recall. Conclusions Our work preliminarily confirmed the validity of attention mechanism in discovering key information and mining text features, which might provide useful ideas for future research in clinical named entity recognition of Chinese electronic medical records. In the future, we will explore the deeper application of attention mechanism in neural network. |
| first_indexed | 2024-12-13T18:51:01Z |
| format | Article |
| id | doaj.art-76c20095bfdc4cf9b7d1505c3ccc2595 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-12-13T18:51:01Z |
| publishDate | 2019-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-76c20095bfdc4cf9b7d1505c3ccc25952022-12-21T23:34:56ZengBMCBMC Medical Informatics and Decision Making1472-69472019-12-0119S511110.1186/s12911-019-0933-6An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical recordsLuqi Li0Jie Zhao1Li Hou2Yunkai Zhai3Jinming Shi4Fangfang Cui5Institute of Medical Information, Chinese Academy of Medical Sciences/Peking Union Medical CollegeNational Engineering Laboratory for Internet Medical Systems and Applications, The First Affiliated Hospital of Zhengzhou UniversityInstitute of Medical Information, Chinese Academy of Medical Sciences/Peking Union Medical CollegeNational Engineering Laboratory for Internet Medical Systems and Applications, The First Affiliated Hospital of Zhengzhou UniversityNational Engineering Laboratory for Internet Medical Systems and Applications, The First Affiliated Hospital of Zhengzhou UniversityNational Engineering Laboratory for Internet Medical Systems and Applications, The First Affiliated Hospital of Zhengzhou UniversityAbstract Background Clinical named entity recognition (CNER) is important for medical information mining and establishment of high-quality knowledge map. Due to the different text features from natural language and a large number of professional and uncommon clinical terms in Chinese electronic medical records (EMRs), there are still many difficulties in clinical named entity recognition of Chinese EMRs. It is of great importance to eliminate semantic interference and improve the ability of autonomous learning of internal features of the model under the small training corpus. Methods From the perspective of deep learning, we integrated the attention mechanism into neural network, and proposed an improved clinical named entity recognition method for Chinese electronic medical records called BiLSTM-Att-CRF, which could capture more useful information of the context and avoid the problem of missing information caused by long-distance factors. In addition, medical dictionaries and part-of-speech (POS) features were also introduced to improve the performance of the model. Results Based on China Conference on Knowledge Graph and Semantic Computing (CCKS) 2017 and 2018 Chinese EMRs corpus, our BiLSTM-Att-CRF model finally achieved better performance than other widely-used models without additional features(F1-measure of 85.4% in CCKS 2018, F1-measure of 90.29% in CCKS 2017), and achieved the best performance with POS and dictionary features (F1-measure of 86.11% in CCKS 2018, F1-measure of 90.48% in CCKS 2017). In particular, the BiLSTM-Att-CRF model had significant effect on the improvement of Recall. Conclusions Our work preliminarily confirmed the validity of attention mechanism in discovering key information and mining text features, which might provide useful ideas for future research in clinical named entity recognition of Chinese electronic medical records. In the future, we will explore the deeper application of attention mechanism in neural network.https://doi.org/10.1186/s12911-019-0933-6Named entity recognitionAttention mechanismChinese electronic medical records |
| spellingShingle | Luqi Li Jie Zhao Li Hou Yunkai Zhai Jinming Shi Fangfang Cui An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records BMC Medical Informatics and Decision Making Named entity recognition Attention mechanism Chinese electronic medical records |
| title | An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records |
| title_full | An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records |
| title_fullStr | An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records |
| title_full_unstemmed | An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records |
| title_short | An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records |
| title_sort | attention based deep learning model for clinical named entity recognition of chinese electronic medical records |
| topic | Named entity recognition Attention mechanism Chinese electronic medical records |
| url | https://doi.org/10.1186/s12911-019-0933-6 |
| work_keys_str_mv | AT luqili anattentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT jiezhao anattentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT lihou anattentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT yunkaizhai anattentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT jinmingshi anattentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT fangfangcui anattentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT luqili attentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT jiezhao attentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT lihou attentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT yunkaizhai attentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT jinmingshi attentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords AT fangfangcui attentionbaseddeeplearningmodelforclinicalnamedentityrecognitionofchineseelectronicmedicalrecords |