Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition
In recent years, a huge amount of text information requires processing to support the diagnosis and treatment of diabetes in the medical field; therefore, the named entity recognition of diabetes (DNER) is giving rise to the popularity of this research topic within this particular field. Although th...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/11/2412 |
| _version_ | 1797597147977220096 |
|---|---|
| author | Lin Shi Xianming Zou Chenxu Dai Zhanlin Ji |
| author_facet | Lin Shi Xianming Zou Chenxu Dai Zhanlin Ji |
| author_sort | Lin Shi |
| collection | DOAJ |
| description | In recent years, a huge amount of text information requires processing to support the diagnosis and treatment of diabetes in the medical field; therefore, the named entity recognition of diabetes (DNER) is giving rise to the popularity of this research topic within this particular field. Although the mainstream methods for Chinese medical named entity recognition can effectively capture global context information, they ignore the potential local information in sentences, and hence cannot extract the local context features through an efficient framework. To overcome these challenges, this paper constructs a diabetes corpus and proposes the RMBC (RoBERTa Multi-scale CNN BiGRU Self-attention CRF) model. This model is a named entity recognition model that unites multi-scale local feature awareness and the self-attention mechanism. This paper first utilizes RoBERTa-wwm to encode the characters; then, it designs a local context-wise module, which captures the context information containing locally important features by fusing multi-window attention with residual convolution at the multi-scale and adds a self-attention mechanism to address the restriction of the bidirectional gated recurrent unit (BiGRU) capturing long-distance dependencies and to obtain global semantic information. Finally, conditional random fields (CRF) are relied on to learn of the dependency between adjacent tags and to obtain the optimal tag sequence. The experimental results on our constructed private dataset, termed DNER, along with two benchmark datasets, demonstrate the effectiveness of the model in this paper. |
| first_indexed | 2024-03-11T03:02:41Z |
| format | Article |
| id | doaj.art-795595ca7d8d4cc2b2c60f10ad73fd70 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T03:02:41Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-795595ca7d8d4cc2b2c60f10ad73fd702023-11-18T08:11:40ZengMDPI AGMathematics2227-73902023-05-011111241210.3390/math11112412Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity RecognitionLin Shi0Xianming Zou1Chenxu Dai2Zhanlin Ji3Hebei Key Laboratory of Industrial Intelligent Perception, College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, ChinaHebei Key Laboratory of Industrial Intelligent Perception, College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, ChinaHebei Key Laboratory of Industrial Intelligent Perception, College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, ChinaHebei Key Laboratory of Industrial Intelligent Perception, College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, ChinaIn recent years, a huge amount of text information requires processing to support the diagnosis and treatment of diabetes in the medical field; therefore, the named entity recognition of diabetes (DNER) is giving rise to the popularity of this research topic within this particular field. Although the mainstream methods for Chinese medical named entity recognition can effectively capture global context information, they ignore the potential local information in sentences, and hence cannot extract the local context features through an efficient framework. To overcome these challenges, this paper constructs a diabetes corpus and proposes the RMBC (RoBERTa Multi-scale CNN BiGRU Self-attention CRF) model. This model is a named entity recognition model that unites multi-scale local feature awareness and the self-attention mechanism. This paper first utilizes RoBERTa-wwm to encode the characters; then, it designs a local context-wise module, which captures the context information containing locally important features by fusing multi-window attention with residual convolution at the multi-scale and adds a self-attention mechanism to address the restriction of the bidirectional gated recurrent unit (BiGRU) capturing long-distance dependencies and to obtain global semantic information. Finally, conditional random fields (CRF) are relied on to learn of the dependency between adjacent tags and to obtain the optimal tag sequence. The experimental results on our constructed private dataset, termed DNER, along with two benchmark datasets, demonstrate the effectiveness of the model in this paper.https://www.mdpi.com/2227-7390/11/11/2412named entity recognitiondiabetes datasetmulti-scale local feature awarenessresidual structureself-attention mechanism |
| spellingShingle | Lin Shi Xianming Zou Chenxu Dai Zhanlin Ji Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition Mathematics named entity recognition diabetes dataset multi-scale local feature awareness residual structure self-attention mechanism |
| title | Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition |
| title_full | Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition |
| title_fullStr | Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition |
| title_full_unstemmed | Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition |
| title_short | Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition |
| title_sort | uniting multi scale local feature awareness and the self attention mechanism for named entity recognition |
| topic | named entity recognition diabetes dataset multi-scale local feature awareness residual structure self-attention mechanism |
| url | https://www.mdpi.com/2227-7390/11/11/2412 |
| work_keys_str_mv | AT linshi unitingmultiscalelocalfeatureawarenessandtheselfattentionmechanismfornamedentityrecognition AT xianmingzou unitingmultiscalelocalfeatureawarenessandtheselfattentionmechanismfornamedentityrecognition AT chenxudai unitingmultiscalelocalfeatureawarenessandtheselfattentionmechanismfornamedentityrecognition AT zhanlinji unitingmultiscalelocalfeatureawarenessandtheselfattentionmechanismfornamedentityrecognition |