Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition
The automatic extraction of key entities in mechanics problems is an important means to automatically solve mechanics problems. Nevertheless, for standard Chinese, compared with the open domain, mechanics problems have a large number of specialized terms and composite entities, which leads to a low...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/20/11325 |
| _version_ | 1797574875445985280 |
|---|---|
| author | Jiarong Zhang Jinsha Yuan Jing Zhang Zhihong Luo Aitong Li |
| author_facet | Jiarong Zhang Jinsha Yuan Jing Zhang Zhihong Luo Aitong Li |
| author_sort | Jiarong Zhang |
| collection | DOAJ |
| description | The automatic extraction of key entities in mechanics problems is an important means to automatically solve mechanics problems. Nevertheless, for standard Chinese, compared with the open domain, mechanics problems have a large number of specialized terms and composite entities, which leads to a low recognition capability. Although recent research demonstrates that external information and pre-trained language models can improve the performance of Chinese Named Entity Recognition (CNER), few efforts have been made to combine the two to explore high-performance algorithms for extracting mechanics entities. Therefore, this article proposes a Multi-Meta Information Embedding Enhanced Bidirectional Encoder Representation from Transformers (MMIEE-BERT) for recognizing entities in mechanics problems. The proposed method integrates lexical information and radical information into BERT layers directly by employing an information adapter layer (IAL). Firstly, according to the characteristics of Chinese, a Multi-Meta Information Embedding (MMIE) including character embedding, lexical embedding, and radical embedding is proposed to enhance Chinese sentence representation. Secondly, an information adapter layer (IAL) is proposed to fuse the above three embeddings into the lower layers of the BERT. Thirdly, a Bidirectional Long Short-Term Memory (BiLSTM) network and a Conditional Random Field (CRF) model are applied to semantically encode the output of MMIEE-BERT and obtain each character’s label. Finally, extensive experiments were carried out on the dataset built by our team and widely used datasets. The results demonstrate that the proposed method has more advantages than the existing models in the entity recognition of mechanics problems, and the precision, recall, and F1 score were improved. The proposed method is expected to provide an automatic means for extracting key information from mechanics problems. |
| first_indexed | 2024-03-10T21:28:18Z |
| format | Article |
| id | doaj.art-502cb26ebafc4ef180ebd2df4e7d0bdd |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T21:28:18Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-502cb26ebafc4ef180ebd2df4e7d0bdd2023-11-19T15:30:39ZengMDPI AGApplied Sciences2076-34172023-10-0113201132510.3390/app132011325Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity RecognitionJiarong Zhang0Jinsha Yuan1Jing Zhang2Zhihong Luo3Aitong Li4Department of Electronic and Communication Engineering, North China Electric Power University, Baoding 071003, ChinaDepartment of Electronic and Communication Engineering, North China Electric Power University, Baoding 071003, ChinaDepartment of New Energy Power Technology Research, COMAC Beijing Aircraft Technology Research Institute, Beijing 102211, ChinaDepartment of Electric Power, Inner Mongolia University of Technology, Hohhot 010051, ChinaCollege of Economics, Bohai University, Jinzhou 121013, ChinaThe automatic extraction of key entities in mechanics problems is an important means to automatically solve mechanics problems. Nevertheless, for standard Chinese, compared with the open domain, mechanics problems have a large number of specialized terms and composite entities, which leads to a low recognition capability. Although recent research demonstrates that external information and pre-trained language models can improve the performance of Chinese Named Entity Recognition (CNER), few efforts have been made to combine the two to explore high-performance algorithms for extracting mechanics entities. Therefore, this article proposes a Multi-Meta Information Embedding Enhanced Bidirectional Encoder Representation from Transformers (MMIEE-BERT) for recognizing entities in mechanics problems. The proposed method integrates lexical information and radical information into BERT layers directly by employing an information adapter layer (IAL). Firstly, according to the characteristics of Chinese, a Multi-Meta Information Embedding (MMIE) including character embedding, lexical embedding, and radical embedding is proposed to enhance Chinese sentence representation. Secondly, an information adapter layer (IAL) is proposed to fuse the above three embeddings into the lower layers of the BERT. Thirdly, a Bidirectional Long Short-Term Memory (BiLSTM) network and a Conditional Random Field (CRF) model are applied to semantically encode the output of MMIEE-BERT and obtain each character’s label. Finally, extensive experiments were carried out on the dataset built by our team and widely used datasets. The results demonstrate that the proposed method has more advantages than the existing models in the entity recognition of mechanics problems, and the precision, recall, and F1 score were improved. The proposed method is expected to provide an automatic means for extracting key information from mechanics problems.https://www.mdpi.com/2076-3417/13/20/11325named entity recognitionMulti-Meta Information EmbeddingBERTinformation adapter layerBiLSTMCRF |
| spellingShingle | Jiarong Zhang Jinsha Yuan Jing Zhang Zhihong Luo Aitong Li Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition Applied Sciences named entity recognition Multi-Meta Information Embedding BERT information adapter layer BiLSTM CRF |
| title | Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition |
| title_full | Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition |
| title_fullStr | Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition |
| title_full_unstemmed | Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition |
| title_short | Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition |
| title_sort | multi meta information embedding enhanced bert for chinese mechanics entity recognition |
| topic | named entity recognition Multi-Meta Information Embedding BERT information adapter layer BiLSTM CRF |
| url | https://www.mdpi.com/2076-3417/13/20/11325 |
| work_keys_str_mv | AT jiarongzhang multimetainformationembeddingenhancedbertforchinesemechanicsentityrecognition AT jinshayuan multimetainformationembeddingenhancedbertforchinesemechanicsentityrecognition AT jingzhang multimetainformationembeddingenhancedbertforchinesemechanicsentityrecognition AT zhihongluo multimetainformationembeddingenhancedbertforchinesemechanicsentityrecognition AT aitongli multimetainformationembeddingenhancedbertforchinesemechanicsentityrecognition |