Converse attention knowledge transfer for low-resource named entity recognition
In recent years, great success has been achieved in many tasks of natural language processing (NLP), e.g., named entity recognition (NER), especially in the high-resource language, i.e., English, thanks in part to the considerable amount of labeled resources. More labeled resources, better word repr...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/181468 |
| _version_ | 1826112086323757056 |
|---|---|
| author | Lyu, Shengfei Sun, Linghao Yi, Huixiong Liu, Yong Chen, Huanhuan Miao, Chunyan |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Lyu, Shengfei Sun, Linghao Yi, Huixiong Liu, Yong Chen, Huanhuan Miao, Chunyan |
| author_sort | Lyu, Shengfei |
| collection | NTU |
| description | In recent years, great success has been achieved in many tasks of natural language processing (NLP), e.g., named entity recognition (NER), especially in the high-resource language, i.e., English, thanks in part to the considerable amount of labeled resources. More labeled resources, better word representations. However, most low-resource languages do not have such an abundance of labeled data as high-resource English, leading to poor performance of NER in these low-resource languages due to poor word representations. In the paper, we propose converse attention network (CAN) to augment word representations in low-resource languages from the high-resource language, improving the performance of NER in low-resource languages by transferring knowledge learned in the high-resource language. CAN first translates sentences in low-resource languages into high-resource English using an attention-based translation module. In the process of translation, CAN obtains the attention matrices that align word representations of high-resource language space and low-resource language space. Furthermore, CAN augments word representations learned in low-resource language space with word representations learned in high-resource language space using the attention matrices. Experiments on four low-resource NER datasets show that CAN achieves consistent and significant performance improvements, which indicates the effectiveness of CAN. |
| first_indexed | 2025-03-09T10:29:55Z |
| format | Journal Article |
| id | ntu-10356/181468 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-03-09T10:29:55Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1814682024-12-06T15:38:29Z Converse attention knowledge transfer for low-resource named entity recognition Lyu, Shengfei Sun, Linghao Yi, Huixiong Liu, Yong Chen, Huanhuan Miao, Chunyan School of Computer Science and Engineering Computer and Information Science Named entity recognition Converse attention network In recent years, great success has been achieved in many tasks of natural language processing (NLP), e.g., named entity recognition (NER), especially in the high-resource language, i.e., English, thanks in part to the considerable amount of labeled resources. More labeled resources, better word representations. However, most low-resource languages do not have such an abundance of labeled data as high-resource English, leading to poor performance of NER in these low-resource languages due to poor word representations. In the paper, we propose converse attention network (CAN) to augment word representations in low-resource languages from the high-resource language, improving the performance of NER in low-resource languages by transferring knowledge learned in the high-resource language. CAN first translates sentences in low-resource languages into high-resource English using an attention-based translation module. In the process of translation, CAN obtains the attention matrices that align word representations of high-resource language space and low-resource language space. Furthermore, CAN augments word representations learned in low-resource language space with word representations learned in high-resource language space using the attention matrices. Experiments on four low-resource NER datasets show that CAN achieves consistent and significant performance improvements, which indicates the effectiveness of CAN. Published version This work was supported in part by the National Key Research and Development Program of China (No. 2021ZD0111700), National Natural Science Foundation of China (Nos. 62206261, 62137002, and 62176245), Key Research and Development Program of Anhui Province (No. 202104a05020011), Key Science and Technology Special Project of Anhui Province (No. 202103a07020002), and Fundamental Research Funds for the Central Universities (No. WK2150110026). 2024-12-03T04:36:52Z 2024-12-03T04:36:52Z 2024 Journal Article Lyu, S., Sun, L., Yi, H., Liu, Y., Chen, H. & Miao, C. (2024). Converse attention knowledge transfer for low-resource named entity recognition. International Journal of Crowd Science, 8(3), 140-148. https://dx.doi.org/10.26599/IJCS.2023.9100014 2398-7294 https://hdl.handle.net/10356/181468 10.26599/IJCS.2023.9100014 2-s2.0-85202930047 3 8 140 148 en International Journal of Crowd Science © The author(s) 2024. The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/). application/pdf |
| spellingShingle | Computer and Information Science Named entity recognition Converse attention network Lyu, Shengfei Sun, Linghao Yi, Huixiong Liu, Yong Chen, Huanhuan Miao, Chunyan Converse attention knowledge transfer for low-resource named entity recognition |
| title | Converse attention knowledge transfer for low-resource named entity recognition |
| title_full | Converse attention knowledge transfer for low-resource named entity recognition |
| title_fullStr | Converse attention knowledge transfer for low-resource named entity recognition |
| title_full_unstemmed | Converse attention knowledge transfer for low-resource named entity recognition |
| title_short | Converse attention knowledge transfer for low-resource named entity recognition |
| title_sort | converse attention knowledge transfer for low resource named entity recognition |
| topic | Computer and Information Science Named entity recognition Converse attention network |
| url | https://hdl.handle.net/10356/181468 |
| work_keys_str_mv | AT lyushengfei converseattentionknowledgetransferforlowresourcenamedentityrecognition AT sunlinghao converseattentionknowledgetransferforlowresourcenamedentityrecognition AT yihuixiong converseattentionknowledgetransferforlowresourcenamedentityrecognition AT liuyong converseattentionknowledgetransferforlowresourcenamedentityrecognition AT chenhuanhuan converseattentionknowledgetransferforlowresourcenamedentityrecognition AT miaochunyan converseattentionknowledgetransferforlowresourcenamedentityrecognition |