Named Entity Recognition Networks Based on Syntactically Constrained Attention
The task of named entity recognition can be transformed into a machine reading comprehension task by associating the query and its context, which contains entity information, with the encoding layer. In this process, the model learns a priori knowledge about the entity, from the query, to achieve go...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/6/3993 |
| _version_ | 1827751409969266688 |
|---|---|
| author | Weiwei Sun Shengquan Liu Yan Liu Lingqi Kong Zhaorui Jian |
| author_facet | Weiwei Sun Shengquan Liu Yan Liu Lingqi Kong Zhaorui Jian |
| author_sort | Weiwei Sun |
| collection | DOAJ |
| description | The task of named entity recognition can be transformed into a machine reading comprehension task by associating the query and its context, which contains entity information, with the encoding layer. In this process, the model learns a priori knowledge about the entity, from the query, to achieve good results. However, as the length of the context and query increases, the model struggles with an increasing number of less relevant words, which can distract it from the task. Although attention mechanisms can help the model understand contextual semantic relations, without explicit constraint information, attention may be allocated to less task-relevant words, leading to a bias in the model’s understanding of the context. To address this problem, we propose a new model, the syntactic constraint-based dual-context aggregation network, which uses syntactic information to guide query and context modeling. By incorporating syntactic constraint information into the attention mechanism, the model can better determine the relevance of each word in the context of the task, and selectively focus on the relevant parts of the context. This enhances the model’s ability to read and understand the context, ultimately improving its performance in named entity recognition tasks. Extensive experiments on three datasets, ACE2004, ACE2005, and GENIA, show that this method achieves superior performance when compared to previous methods. |
| first_indexed | 2024-03-11T06:57:47Z |
| format | Article |
| id | doaj.art-9495ea2158f449e9ae974e4f39c4de8e |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T06:57:47Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-9495ea2158f449e9ae974e4f39c4de8e2023-11-17T09:29:56ZengMDPI AGApplied Sciences2076-34172023-03-01136399310.3390/app13063993Named Entity Recognition Networks Based on Syntactically Constrained AttentionWeiwei Sun0Shengquan Liu1Yan Liu2Lingqi Kong3Zhaorui Jian4College of Information Science and Engineering, Xinjiang University, Urumqi 830046, ChinaCollege of Information Science and Engineering, Xinjiang University, Urumqi 830046, ChinaCollege of Information Science and Engineering, Xinjiang University, Urumqi 830046, ChinaCollege of Information Science and Engineering, Xinjiang University, Urumqi 830046, ChinaCollege of Information Science and Engineering, Xinjiang University, Urumqi 830046, ChinaThe task of named entity recognition can be transformed into a machine reading comprehension task by associating the query and its context, which contains entity information, with the encoding layer. In this process, the model learns a priori knowledge about the entity, from the query, to achieve good results. However, as the length of the context and query increases, the model struggles with an increasing number of less relevant words, which can distract it from the task. Although attention mechanisms can help the model understand contextual semantic relations, without explicit constraint information, attention may be allocated to less task-relevant words, leading to a bias in the model’s understanding of the context. To address this problem, we propose a new model, the syntactic constraint-based dual-context aggregation network, which uses syntactic information to guide query and context modeling. By incorporating syntactic constraint information into the attention mechanism, the model can better determine the relevance of each word in the context of the task, and selectively focus on the relevant parts of the context. This enhances the model’s ability to read and understand the context, ultimately improving its performance in named entity recognition tasks. Extensive experiments on three datasets, ACE2004, ACE2005, and GENIA, show that this method achieves superior performance when compared to previous methods.https://www.mdpi.com/2076-3417/13/6/3993named entity recognitionmachine reading comprehensionsyntactic constraint information |
| spellingShingle | Weiwei Sun Shengquan Liu Yan Liu Lingqi Kong Zhaorui Jian Named Entity Recognition Networks Based on Syntactically Constrained Attention Applied Sciences named entity recognition machine reading comprehension syntactic constraint information |
| title | Named Entity Recognition Networks Based on Syntactically Constrained Attention |
| title_full | Named Entity Recognition Networks Based on Syntactically Constrained Attention |
| title_fullStr | Named Entity Recognition Networks Based on Syntactically Constrained Attention |
| title_full_unstemmed | Named Entity Recognition Networks Based on Syntactically Constrained Attention |
| title_short | Named Entity Recognition Networks Based on Syntactically Constrained Attention |
| title_sort | named entity recognition networks based on syntactically constrained attention |
| topic | named entity recognition machine reading comprehension syntactic constraint information |
| url | https://www.mdpi.com/2076-3417/13/6/3993 |
| work_keys_str_mv | AT weiweisun namedentityrecognitionnetworksbasedonsyntacticallyconstrainedattention AT shengquanliu namedentityrecognitionnetworksbasedonsyntacticallyconstrainedattention AT yanliu namedentityrecognitionnetworksbasedonsyntacticallyconstrainedattention AT lingqikong namedentityrecognitionnetworksbasedonsyntacticallyconstrainedattention AT zhaoruijian namedentityrecognitionnetworksbasedonsyntacticallyconstrainedattention |