Mixup Based Cross-Consistency Training for Named Entity Recognition
Named Entity Recognition (NER) is at the core of natural language understanding. The quality and amount of datasets determine the performance of deep-learning-based NER models. As datasets for NER require token-level or word-level labels to be assigned, annotating the datasets is expensive and time...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/21/11084 |
| _version_ | 1797469230576173056 |
|---|---|
| author | Geonsik Youn Bohan Yoon Seungbin Ji Dahee Ko Jongtae Rhee |
| author_facet | Geonsik Youn Bohan Yoon Seungbin Ji Dahee Ko Jongtae Rhee |
| author_sort | Geonsik Youn |
| collection | DOAJ |
| description | Named Entity Recognition (NER) is at the core of natural language understanding. The quality and amount of datasets determine the performance of deep-learning-based NER models. As datasets for NER require token-level or word-level labels to be assigned, annotating the datasets is expensive and time consuming. To alleviate efforts of manual anotation, many prior studies utilized weak supervision for NER tasks. However, using weak supervision directly would be an obstacle for training deep networks because the labels automatically annotated contain a a lot of noise. In this study, we propose a framework to better train the deep model for NER tasks using weakly labeled data. The proposed framework stems from the idea that mixup, which was recently considered as a data augmentation strategy, would be an obstacle to deep model training for NER tasks. Inspired by this idea, we used mixup as a perturbation function for consistency regularization, one of the semi-supervised learning strategies. To support our idea, we conducted several experiments for NER benchmarks. Experimental results proved that directly using mixup on NER tasks hinders deep model training while demonstrating that the proposed framework achieves improved performances compared to employing only a few human-annotated data. |
| first_indexed | 2024-03-09T19:18:26Z |
| format | Article |
| id | doaj.art-b9f8d9d68f064c22a7899fb170ee5939 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T19:18:26Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-b9f8d9d68f064c22a7899fb170ee59392023-11-24T03:37:47ZengMDPI AGApplied Sciences2076-34172022-11-0112211108410.3390/app122111084Mixup Based Cross-Consistency Training for Named Entity RecognitionGeonsik Youn0Bohan Yoon1Seungbin Ji2Dahee Ko3Jongtae Rhee4Department of Industrial and Systems Engineering, Dongguk University, Seoul 04620, KoreaDepartment of Industrial and Systems Engineering, Dongguk University, Seoul 04620, KoreaDepartment of Industrial and Systems Engineering, Dongguk University, Seoul 04620, KoreaDepartment of Industrial and Systems Engineering, Dongguk University, Seoul 04620, KoreaDepartment of Industrial and Systems Engineering, Dongguk University, Seoul 04620, KoreaNamed Entity Recognition (NER) is at the core of natural language understanding. The quality and amount of datasets determine the performance of deep-learning-based NER models. As datasets for NER require token-level or word-level labels to be assigned, annotating the datasets is expensive and time consuming. To alleviate efforts of manual anotation, many prior studies utilized weak supervision for NER tasks. However, using weak supervision directly would be an obstacle for training deep networks because the labels automatically annotated contain a a lot of noise. In this study, we propose a framework to better train the deep model for NER tasks using weakly labeled data. The proposed framework stems from the idea that mixup, which was recently considered as a data augmentation strategy, would be an obstacle to deep model training for NER tasks. Inspired by this idea, we used mixup as a perturbation function for consistency regularization, one of the semi-supervised learning strategies. To support our idea, we conducted several experiments for NER benchmarks. Experimental results proved that directly using mixup on NER tasks hinders deep model training while demonstrating that the proposed framework achieves improved performances compared to employing only a few human-annotated data.https://www.mdpi.com/2076-3417/12/21/11084deep learningnamed entity recognitionconsistency regularizationsemi-supervised learningmixup |
| spellingShingle | Geonsik Youn Bohan Yoon Seungbin Ji Dahee Ko Jongtae Rhee Mixup Based Cross-Consistency Training for Named Entity Recognition Applied Sciences deep learning named entity recognition consistency regularization semi-supervised learning mixup |
| title | Mixup Based Cross-Consistency Training for Named Entity Recognition |
| title_full | Mixup Based Cross-Consistency Training for Named Entity Recognition |
| title_fullStr | Mixup Based Cross-Consistency Training for Named Entity Recognition |
| title_full_unstemmed | Mixup Based Cross-Consistency Training for Named Entity Recognition |
| title_short | Mixup Based Cross-Consistency Training for Named Entity Recognition |
| title_sort | mixup based cross consistency training for named entity recognition |
| topic | deep learning named entity recognition consistency regularization semi-supervised learning mixup |
| url | https://www.mdpi.com/2076-3417/12/21/11084 |
| work_keys_str_mv | AT geonsikyoun mixupbasedcrossconsistencytrainingfornamedentityrecognition AT bohanyoon mixupbasedcrossconsistencytrainingfornamedentityrecognition AT seungbinji mixupbasedcrossconsistencytrainingfornamedentityrecognition AT daheeko mixupbasedcrossconsistencytrainingfornamedentityrecognition AT jongtaerhee mixupbasedcrossconsistencytrainingfornamedentityrecognition |