Deep learning with language models improves named entity recognition for PharmaCoNER
Abstract Background The recognition of pharmacological substances, compounds and proteins is essential for biomedical relation extraction, knowledge graph construction, drug discovery, as well as medical question answering. Although considerable efforts have been made to recognize biomedical entitie...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2021-12-01
|
| Series: | BMC Bioinformatics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12859-021-04260-y |
| _version_ | 1797397731219603456 |
|---|---|
| author | Cong Sun Zhihao Yang Lei Wang Yin Zhang Hongfei Lin Jian Wang |
| author_facet | Cong Sun Zhihao Yang Lei Wang Yin Zhang Hongfei Lin Jian Wang |
| author_sort | Cong Sun |
| collection | DOAJ |
| description | Abstract Background The recognition of pharmacological substances, compounds and proteins is essential for biomedical relation extraction, knowledge graph construction, drug discovery, as well as medical question answering. Although considerable efforts have been made to recognize biomedical entities in English texts, to date, only few limited attempts were made to recognize them from biomedical texts in other languages. PharmaCoNER is a named entity recognition challenge to recognize pharmacological entities from Spanish texts. Because there are currently abundant resources in the field of natural language processing, how to leverage these resources to the PharmaCoNER challenge is a meaningful study. Methods Inspired by the success of deep learning with language models, we compare and explore various representative BERT models to promote the development of the PharmaCoNER task. Results The experimental results show that deep learning with language models can effectively improve model performance on the PharmaCoNER dataset. Our method achieves state-of-the-art performance on the PharmaCoNER dataset, with a max F1-score of 92.01%. Conclusion For the BERT models on the PharmaCoNER dataset, biomedical domain knowledge has a greater impact on model performance than the native language (i.e., Spanish). The BERT models can obtain competitive performance by using WordPiece to alleviate the out of vocabulary limitation. The performance on the BERT model can be further improved by constructing a specific vocabulary based on domain knowledge. Moreover, the character case also has a certain impact on model performance. |
| first_indexed | 2024-03-09T01:14:25Z |
| format | Article |
| id | doaj.art-3276f25b7a404fe5a39e46425fc733ef |
| institution | Directory Open Access Journal |
| issn | 1471-2105 |
| language | English |
| last_indexed | 2024-03-09T01:14:25Z |
| publishDate | 2021-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Bioinformatics |
| spelling | doaj.art-3276f25b7a404fe5a39e46425fc733ef2023-12-10T12:33:50ZengBMCBMC Bioinformatics1471-21052021-12-0122S111610.1186/s12859-021-04260-yDeep learning with language models improves named entity recognition for PharmaCoNERCong Sun0Zhihao Yang1Lei Wang2Yin Zhang3Hongfei Lin4Jian Wang5School of Computer Science and Technology, Dalian University of TechnologySchool of Computer Science and Technology, Dalian University of TechnologyBeijing Institute of Health Administration and Medical InformationBeijing Institute of Health Administration and Medical InformationSchool of Computer Science and Technology, Dalian University of TechnologySchool of Computer Science and Technology, Dalian University of TechnologyAbstract Background The recognition of pharmacological substances, compounds and proteins is essential for biomedical relation extraction, knowledge graph construction, drug discovery, as well as medical question answering. Although considerable efforts have been made to recognize biomedical entities in English texts, to date, only few limited attempts were made to recognize them from biomedical texts in other languages. PharmaCoNER is a named entity recognition challenge to recognize pharmacological entities from Spanish texts. Because there are currently abundant resources in the field of natural language processing, how to leverage these resources to the PharmaCoNER challenge is a meaningful study. Methods Inspired by the success of deep learning with language models, we compare and explore various representative BERT models to promote the development of the PharmaCoNER task. Results The experimental results show that deep learning with language models can effectively improve model performance on the PharmaCoNER dataset. Our method achieves state-of-the-art performance on the PharmaCoNER dataset, with a max F1-score of 92.01%. Conclusion For the BERT models on the PharmaCoNER dataset, biomedical domain knowledge has a greater impact on model performance than the native language (i.e., Spanish). The BERT models can obtain competitive performance by using WordPiece to alleviate the out of vocabulary limitation. The performance on the BERT model can be further improved by constructing a specific vocabulary based on domain knowledge. Moreover, the character case also has a certain impact on model performance.https://doi.org/10.1186/s12859-021-04260-yNamed entity recognitionNERLanguage modelBERTText mining |
| spellingShingle | Cong Sun Zhihao Yang Lei Wang Yin Zhang Hongfei Lin Jian Wang Deep learning with language models improves named entity recognition for PharmaCoNER BMC Bioinformatics Named entity recognition NER Language model BERT Text mining |
| title | Deep learning with language models improves named entity recognition for PharmaCoNER |
| title_full | Deep learning with language models improves named entity recognition for PharmaCoNER |
| title_fullStr | Deep learning with language models improves named entity recognition for PharmaCoNER |
| title_full_unstemmed | Deep learning with language models improves named entity recognition for PharmaCoNER |
| title_short | Deep learning with language models improves named entity recognition for PharmaCoNER |
| title_sort | deep learning with language models improves named entity recognition for pharmaconer |
| topic | Named entity recognition NER Language model BERT Text mining |
| url | https://doi.org/10.1186/s12859-021-04260-y |
| work_keys_str_mv | AT congsun deeplearningwithlanguagemodelsimprovesnamedentityrecognitionforpharmaconer AT zhihaoyang deeplearningwithlanguagemodelsimprovesnamedentityrecognitionforpharmaconer AT leiwang deeplearningwithlanguagemodelsimprovesnamedentityrecognitionforpharmaconer AT yinzhang deeplearningwithlanguagemodelsimprovesnamedentityrecognitionforpharmaconer AT hongfeilin deeplearningwithlanguagemodelsimprovesnamedentityrecognitionforpharmaconer AT jianwang deeplearningwithlanguagemodelsimprovesnamedentityrecognitionforpharmaconer |