A Multi-Granularity Word Fusion Method for Chinese NER
Named entity recognition (NER) plays a crucial role in many downstream natural language processing (NLP) tasks. It is challenging for Chinese NER because of certain features of Chinese. Recently, large-scaled pre-training language models have been used in Chinese NER. However, since some of the pre-...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/5/2789 |
| _version_ | 1797615828758167552 |
|---|---|
| author | Tong Liu Jian Gao Weijian Ni Qingtian Zeng |
| author_facet | Tong Liu Jian Gao Weijian Ni Qingtian Zeng |
| author_sort | Tong Liu |
| collection | DOAJ |
| description | Named entity recognition (NER) plays a crucial role in many downstream natural language processing (NLP) tasks. It is challenging for Chinese NER because of certain features of Chinese. Recently, large-scaled pre-training language models have been used in Chinese NER. However, since some of the pre-training language models do not use word information or just employ word information of single granularity, the semantic information in sentences could not be fully captured, which affects these models’ performance. To fully take advantage of word information and obtain richer semantic information, we propose a multi-granularity word fusion method for Chinese NER. We introduce multi-granularity word information into our model. To make full use of the information, we classify the information into three kinds: strong information, moderate information, and weak information. These kinds of information are encoded by encoders and then integrated with each other through the strong-weak feedback attention mechanism. Specifically, we apply two separate attention networks to word embeddings and N-grams embeddings. Then, the outputs are fused into another attention. In these three attentions, character embeddings are used to be the query of attentions. We call the results the multi-granularity word information. To combine character information and multi-granularity word information, we introduce two fusion strategies for better performance. The process makes our model obtain rich semantic information and reduces word segmentation errors and noise in an explicit way. We design experiments to get our model’s best performance by comparing some components. Ablation study is used to verify the effectiveness of each module. The final experiments are conducted on four Chinese NER benchmark datasets and the F1 scores are 81.51% for Ontonotes4.0, 95.47% for MSRA, 95.87% for Resume, and 69.41% for Weibo. The best improvement achieved by the proposed method is 1.37%. Experimental results show that our method outperforms most baselines and achieves the state-of-the-art method in performance. |
| first_indexed | 2024-03-11T07:32:26Z |
| format | Article |
| id | doaj.art-fc0c4658ad924d5ca89c1d31fac46450 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T07:32:26Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-fc0c4658ad924d5ca89c1d31fac464502023-11-17T07:15:04ZengMDPI AGApplied Sciences2076-34172023-02-01135278910.3390/app13052789A Multi-Granularity Word Fusion Method for Chinese NERTong Liu0Jian Gao1Weijian Ni2Qingtian Zeng3College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590, ChinaCollege of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590, ChinaCollege of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590, ChinaCollege of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590, ChinaNamed entity recognition (NER) plays a crucial role in many downstream natural language processing (NLP) tasks. It is challenging for Chinese NER because of certain features of Chinese. Recently, large-scaled pre-training language models have been used in Chinese NER. However, since some of the pre-training language models do not use word information or just employ word information of single granularity, the semantic information in sentences could not be fully captured, which affects these models’ performance. To fully take advantage of word information and obtain richer semantic information, we propose a multi-granularity word fusion method for Chinese NER. We introduce multi-granularity word information into our model. To make full use of the information, we classify the information into three kinds: strong information, moderate information, and weak information. These kinds of information are encoded by encoders and then integrated with each other through the strong-weak feedback attention mechanism. Specifically, we apply two separate attention networks to word embeddings and N-grams embeddings. Then, the outputs are fused into another attention. In these three attentions, character embeddings are used to be the query of attentions. We call the results the multi-granularity word information. To combine character information and multi-granularity word information, we introduce two fusion strategies for better performance. The process makes our model obtain rich semantic information and reduces word segmentation errors and noise in an explicit way. We design experiments to get our model’s best performance by comparing some components. Ablation study is used to verify the effectiveness of each module. The final experiments are conducted on four Chinese NER benchmark datasets and the F1 scores are 81.51% for Ontonotes4.0, 95.47% for MSRA, 95.87% for Resume, and 69.41% for Weibo. The best improvement achieved by the proposed method is 1.37%. Experimental results show that our method outperforms most baselines and achieves the state-of-the-art method in performance.https://www.mdpi.com/2076-3417/13/5/2789Chinese NERcharacter word fusionN-gramsBERT-based modelattention mechanism |
| spellingShingle | Tong Liu Jian Gao Weijian Ni Qingtian Zeng A Multi-Granularity Word Fusion Method for Chinese NER Applied Sciences Chinese NER character word fusion N-grams BERT-based model attention mechanism |
| title | A Multi-Granularity Word Fusion Method for Chinese NER |
| title_full | A Multi-Granularity Word Fusion Method for Chinese NER |
| title_fullStr | A Multi-Granularity Word Fusion Method for Chinese NER |
| title_full_unstemmed | A Multi-Granularity Word Fusion Method for Chinese NER |
| title_short | A Multi-Granularity Word Fusion Method for Chinese NER |
| title_sort | multi granularity word fusion method for chinese ner |
| topic | Chinese NER character word fusion N-grams BERT-based model attention mechanism |
| url | https://www.mdpi.com/2076-3417/13/5/2789 |
| work_keys_str_mv | AT tongliu amultigranularitywordfusionmethodforchinesener AT jiangao amultigranularitywordfusionmethodforchinesener AT weijianni amultigranularitywordfusionmethodforchinesener AT qingtianzeng amultigranularitywordfusionmethodforchinesener AT tongliu multigranularitywordfusionmethodforchinesener AT jiangao multigranularitywordfusionmethodforchinesener AT weijianni multigranularitywordfusionmethodforchinesener AT qingtianzeng multigranularitywordfusionmethodforchinesener |