Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification
Compared with the traditional few-shot task, the few-shot none-of-the-above (NOTA) relation classification focuses on the realistic scenario of few-shot learning, in which a test instance might not belong to any of the target categories. This undoubtedly increases the task’s difficulty because given...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/4/2185 |
| _version_ | 1797482957511852032 |
|---|---|
| author | Bo Lv Li Jin Yanan Zhang Hao Wang Xiaoyu Li Zhi Guo |
| author_facet | Bo Lv Li Jin Yanan Zhang Hao Wang Xiaoyu Li Zhi Guo |
| author_sort | Bo Lv |
| collection | DOAJ |
| description | Compared with the traditional few-shot task, the few-shot none-of-the-above (NOTA) relation classification focuses on the realistic scenario of few-shot learning, in which a test instance might not belong to any of the target categories. This undoubtedly increases the task’s difficulty because given only a few support samples, this cannot represent the distribution of NOTA categories in space. The model needs to make full use of the syntactic information and word meaning information learned in the pre-training stage to distinguish the NOTA category and the support sample category in the embedding space. However, previous fine-tuning methods mainly focus on optimizing the extra classifiers (on top of pre-trained language models (PLMs)) and neglect the connection between pre-training objectives and downstream tasks. In this paper, we propose the commonsense knowledge-aware prompt tuning (CKPT) method for a few-shot NOTA relation classification task. First, a simple and effective prompt-learning method is developed by constructing relation-oriented templates, which can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Second, external knowledge is incorporated into the model by a label-extension operation, which forms knowledgeable prompt tuning to improve and stabilize prompt tuning. Third, to distinguish the NOTA pairs and positive pairs in embedding space more accurately, a learned scoring strategy is proposed, which introduces a learned threshold classification function and improves the loss function by adding a new term focused on NOTA identification. Experiments on two widely used benchmarks (FewRel 2.0 and Few-shot TACRED) show that our method is a simple and effective framework, and a new state of the art is established in the few-shot classification field. |
| first_indexed | 2024-03-09T22:40:57Z |
| format | Article |
| id | doaj.art-c1bb7e0bdfdb42f29a6badb70245584f |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T22:40:57Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-c1bb7e0bdfdb42f29a6badb70245584f2023-11-23T18:40:39ZengMDPI AGApplied Sciences2076-34172022-02-01124218510.3390/app12042185Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation ClassificationBo Lv0Li Jin1Yanan Zhang2Hao Wang3Xiaoyu Li4Zhi Guo5Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaSchool of Information Science and Technology, North China University of Technology, Beijing 100190, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaCompared with the traditional few-shot task, the few-shot none-of-the-above (NOTA) relation classification focuses on the realistic scenario of few-shot learning, in which a test instance might not belong to any of the target categories. This undoubtedly increases the task’s difficulty because given only a few support samples, this cannot represent the distribution of NOTA categories in space. The model needs to make full use of the syntactic information and word meaning information learned in the pre-training stage to distinguish the NOTA category and the support sample category in the embedding space. However, previous fine-tuning methods mainly focus on optimizing the extra classifiers (on top of pre-trained language models (PLMs)) and neglect the connection between pre-training objectives and downstream tasks. In this paper, we propose the commonsense knowledge-aware prompt tuning (CKPT) method for a few-shot NOTA relation classification task. First, a simple and effective prompt-learning method is developed by constructing relation-oriented templates, which can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Second, external knowledge is incorporated into the model by a label-extension operation, which forms knowledgeable prompt tuning to improve and stabilize prompt tuning. Third, to distinguish the NOTA pairs and positive pairs in embedding space more accurately, a learned scoring strategy is proposed, which introduces a learned threshold classification function and improves the loss function by adding a new term focused on NOTA identification. Experiments on two widely used benchmarks (FewRel 2.0 and Few-shot TACRED) show that our method is a simple and effective framework, and a new state of the art is established in the few-shot classification field.https://www.mdpi.com/2076-3417/12/4/2185commonsense knowledge-aware prompt tuningfew-shot none-of-the-above relation classificationpre-trained language modelsscoring strategy |
| spellingShingle | Bo Lv Li Jin Yanan Zhang Hao Wang Xiaoyu Li Zhi Guo Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification Applied Sciences commonsense knowledge-aware prompt tuning few-shot none-of-the-above relation classification pre-trained language models scoring strategy |
| title | Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification |
| title_full | Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification |
| title_fullStr | Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification |
| title_full_unstemmed | Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification |
| title_short | Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification |
| title_sort | commonsense knowledge aware prompt tuning for few shot nota relation classification |
| topic | commonsense knowledge-aware prompt tuning few-shot none-of-the-above relation classification pre-trained language models scoring strategy |
| url | https://www.mdpi.com/2076-3417/12/4/2185 |
| work_keys_str_mv | AT bolv commonsenseknowledgeawareprompttuningforfewshotnotarelationclassification AT lijin commonsenseknowledgeawareprompttuningforfewshotnotarelationclassification AT yananzhang commonsenseknowledgeawareprompttuningforfewshotnotarelationclassification AT haowang commonsenseknowledgeawareprompttuningforfewshotnotarelationclassification AT xiaoyuli commonsenseknowledgeawareprompttuningforfewshotnotarelationclassification AT zhiguo commonsenseknowledgeawareprompttuningforfewshotnotarelationclassification |