Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT
Abstract Background Named entity recognition (NER) of electronic medical records is an important task in clinical medical research. Although deep learning combined with pretraining models performs well in recognizing entities in clinical texts, because Chinese electronic medical records have a speci...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-12-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12911-022-02059-2 |
| _version_ | 1797985519104163840 |
|---|---|
| author | Peng Chen Meng Zhang Xiaosheng Yu Songpu Li |
| author_facet | Peng Chen Meng Zhang Xiaosheng Yu Songpu Li |
| author_sort | Peng Chen |
| collection | DOAJ |
| description | Abstract Background Named entity recognition (NER) of electronic medical records is an important task in clinical medical research. Although deep learning combined with pretraining models performs well in recognizing entities in clinical texts, because Chinese electronic medical records have a special text structure and vocabulary distribution, general pretraining models cannot effectively incorporate entities and medical domain knowledge into representation learning; separate deep network models lack the ability to fully extract rich features in complex texts, which negatively affects the named entity recognition of electronic medical records. Methods To better represent electronic medical record text, we extract the text’s local features and multilevel sequence interaction information to improve the effectiveness of electronic medical record named entity recognition. This paper proposes a hybrid neural network model based on medical MC-BERT, namely, the MC-BERT + BiLSTM + CNN + MHA + CRF model. First, MC-BERT is used as the word embedding model of the text to obtain the word vector, and then BiLSTM and CNN obtain the feature information of the forward and backward directions of the word vector and the local context to obtain the corresponding feature vector. After merging the two feature vectors, they are sent to multihead self-attention (MHA) to obtain multilevel semantic features, and finally, CRF is used to decode the features and predict the label sequence. Results The experiments show that the F1 values of our proposed hybrid neural network model based on MC-BERT reach 94.22%, 86.47%, and 92.28% on the CCKS-2017, CCKS-2019 and cEHRNER datasets, respectively. Compared with the general-domain BERT-based BiLSTM + CRF, our F1 values increased by 0.89%, 1.65% and 2.63%. Finally, we analyzed the effect of an unbalanced number of entities in the electronic medical records on the results of the NER experiment. |
| first_indexed | 2024-04-11T07:19:26Z |
| format | Article |
| id | doaj.art-fc876855819d4665a4758b0a00696ea4 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-04-11T07:19:26Z |
| publishDate | 2022-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-fc876855819d4665a4758b0a00696ea42022-12-22T04:37:49ZengBMCBMC Medical Informatics and Decision Making1472-69472022-12-0122111310.1186/s12911-022-02059-2Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERTPeng Chen0Meng Zhang1Xiaosheng Yu2Songpu Li3College of Computer and Information Technology, China Three Gorges UniversityCollege of Computer and Information Technology, China Three Gorges UniversityCollege of Computer and Information Technology, China Three Gorges UniversityCollege of Economics and Management, China Three Gorges UniversityAbstract Background Named entity recognition (NER) of electronic medical records is an important task in clinical medical research. Although deep learning combined with pretraining models performs well in recognizing entities in clinical texts, because Chinese electronic medical records have a special text structure and vocabulary distribution, general pretraining models cannot effectively incorporate entities and medical domain knowledge into representation learning; separate deep network models lack the ability to fully extract rich features in complex texts, which negatively affects the named entity recognition of electronic medical records. Methods To better represent electronic medical record text, we extract the text’s local features and multilevel sequence interaction information to improve the effectiveness of electronic medical record named entity recognition. This paper proposes a hybrid neural network model based on medical MC-BERT, namely, the MC-BERT + BiLSTM + CNN + MHA + CRF model. First, MC-BERT is used as the word embedding model of the text to obtain the word vector, and then BiLSTM and CNN obtain the feature information of the forward and backward directions of the word vector and the local context to obtain the corresponding feature vector. After merging the two feature vectors, they are sent to multihead self-attention (MHA) to obtain multilevel semantic features, and finally, CRF is used to decode the features and predict the label sequence. Results The experiments show that the F1 values of our proposed hybrid neural network model based on MC-BERT reach 94.22%, 86.47%, and 92.28% on the CCKS-2017, CCKS-2019 and cEHRNER datasets, respectively. Compared with the general-domain BERT-based BiLSTM + CRF, our F1 values increased by 0.89%, 1.65% and 2.63%. Finally, we analyzed the effect of an unbalanced number of entities in the electronic medical records on the results of the NER experiment.https://doi.org/10.1186/s12911-022-02059-2Named entity recognitionBERT modelChinese electronic medical recordHybrid neural network |
| spellingShingle | Peng Chen Meng Zhang Xiaosheng Yu Songpu Li Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT BMC Medical Informatics and Decision Making Named entity recognition BERT model Chinese electronic medical record Hybrid neural network |
| title | Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT |
| title_full | Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT |
| title_fullStr | Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT |
| title_full_unstemmed | Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT |
| title_short | Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT |
| title_sort | named entity recognition of chinese electronic medical records based on a hybrid neural network and medical mc bert |
| topic | Named entity recognition BERT model Chinese electronic medical record Hybrid neural network |
| url | https://doi.org/10.1186/s12911-022-02059-2 |
| work_keys_str_mv | AT pengchen namedentityrecognitionofchineseelectronicmedicalrecordsbasedonahybridneuralnetworkandmedicalmcbert AT mengzhang namedentityrecognitionofchineseelectronicmedicalrecordsbasedonahybridneuralnetworkandmedicalmcbert AT xiaoshengyu namedentityrecognitionofchineseelectronicmedicalrecordsbasedonahybridneuralnetworkandmedicalmcbert AT songpuli namedentityrecognitionofchineseelectronicmedicalrecordsbasedonahybridneuralnetworkandmedicalmcbert |