Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism
Entity and relation extraction has been widely studied in natural language processing, and some joint methods have been proposed in recent years. However, existing studies still suffer from two problems. Firstly, the token space information has been fully utilized in those studies, while the label s...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2022-12-01
|
| Series: | Connection Science |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/09540091.2022.2026295 |
| _version_ | 1797684012356993024 |
|---|---|
| author | Wei Zhao Shan Zhao Shuhui Chen Tien-Hsiung Weng WenJie Kang |
| author_facet | Wei Zhao Shan Zhao Shuhui Chen Tien-Hsiung Weng WenJie Kang |
| author_sort | Wei Zhao |
| collection | DOAJ |
| description | Entity and relation extraction has been widely studied in natural language processing, and some joint methods have been proposed in recent years. However, existing studies still suffer from two problems. Firstly, the token space information has been fully utilized in those studies, while the label space information is underutilized. However, a few preliminary works have proven that the label space information could contribute to this task. Secondly, the performance of relevant entities detection is still unsatisfactory in entity and relation extraction tasks. In this paper, a new model GANCE (Gated and Attentive Network Collaborative Extracting) is proposed to address these problems. Firstly, GANCE exploits the label space information by applying a gating mechanism, which could improve the performance of the relation extraction. Then, two multi-head attention modules are designed to update the token and token-label fusion representation. In this way, the relevant entities detection could be solved. Experimental results demonstrate that GANCE has better accuracy than several competitive approaches in terms of entity recognition and relation extraction on the CoNLL04 dataset at 90.32% and 73.59%, respectively. Moreover, the F1 score of relation extraction increased by 1.24% over existing approaches in the ADE dataset. |
| first_indexed | 2024-03-12T00:24:24Z |
| format | Article |
| id | doaj.art-8d2f941d4fa34648a0f6c0ae2036d491 |
| institution | Directory Open Access Journal |
| issn | 0954-0091 1360-0494 |
| language | English |
| last_indexed | 2024-03-12T00:24:24Z |
| publishDate | 2022-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Connection Science |
| spelling | doaj.art-8d2f941d4fa34648a0f6c0ae2036d4912023-09-15T10:48:00ZengTaylor & Francis GroupConnection Science0954-00911360-04942022-12-0134167068610.1080/09540091.2022.20262952026295Entity and relation collaborative extraction approach based on multi-head attention and gated mechanismWei Zhao0Shan Zhao1Shuhui Chen2Tien-Hsiung Weng3WenJie Kang4National University of Defense TechnologyNational University of Defense TechnologyNational University of Defense TechnologyComputer Science and Information Engineering Providence University TaiwanHunan Police AcademyEntity and relation extraction has been widely studied in natural language processing, and some joint methods have been proposed in recent years. However, existing studies still suffer from two problems. Firstly, the token space information has been fully utilized in those studies, while the label space information is underutilized. However, a few preliminary works have proven that the label space information could contribute to this task. Secondly, the performance of relevant entities detection is still unsatisfactory in entity and relation extraction tasks. In this paper, a new model GANCE (Gated and Attentive Network Collaborative Extracting) is proposed to address these problems. Firstly, GANCE exploits the label space information by applying a gating mechanism, which could improve the performance of the relation extraction. Then, two multi-head attention modules are designed to update the token and token-label fusion representation. In this way, the relevant entities detection could be solved. Experimental results demonstrate that GANCE has better accuracy than several competitive approaches in terms of entity recognition and relation extraction on the CoNLL04 dataset at 90.32% and 73.59%, respectively. Moreover, the F1 score of relation extraction increased by 1.24% over existing approaches in the ADE dataset.http://dx.doi.org/10.1080/09540091.2022.2026295multi-head attentionjoint entity and relation extractiongating mechanismhomoscedastic uncertainty |
| spellingShingle | Wei Zhao Shan Zhao Shuhui Chen Tien-Hsiung Weng WenJie Kang Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism Connection Science multi-head attention joint entity and relation extraction gating mechanism homoscedastic uncertainty |
| title | Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism |
| title_full | Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism |
| title_fullStr | Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism |
| title_full_unstemmed | Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism |
| title_short | Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism |
| title_sort | entity and relation collaborative extraction approach based on multi head attention and gated mechanism |
| topic | multi-head attention joint entity and relation extraction gating mechanism homoscedastic uncertainty |
| url | http://dx.doi.org/10.1080/09540091.2022.2026295 |
| work_keys_str_mv | AT weizhao entityandrelationcollaborativeextractionapproachbasedonmultiheadattentionandgatedmechanism AT shanzhao entityandrelationcollaborativeextractionapproachbasedonmultiheadattentionandgatedmechanism AT shuhuichen entityandrelationcollaborativeextractionapproachbasedonmultiheadattentionandgatedmechanism AT tienhsiungweng entityandrelationcollaborativeextractionapproachbasedonmultiheadattentionandgatedmechanism AT wenjiekang entityandrelationcollaborativeextractionapproachbasedonmultiheadattentionandgatedmechanism |