Intermediate-Task Transfer Learning with BERT for Sarcasm Detection
Sarcasm detection plays an important role in natural language processing as it can impact the performance of many applications, including sentiment analysis, opinion mining, and stance detection. Despite substantial progress on sarcasm detection, the research results are scattered across datasets an...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/10/5/844 |
| _version_ | 1797474480109387776 |
|---|---|
| author | Edoardo Savini Cornelia Caragea |
| author_facet | Edoardo Savini Cornelia Caragea |
| author_sort | Edoardo Savini |
| collection | DOAJ |
| description | Sarcasm detection plays an important role in natural language processing as it can impact the performance of many applications, including sentiment analysis, opinion mining, and stance detection. Despite substantial progress on sarcasm detection, the research results are scattered across datasets and studies. In this paper, we survey the current state-of-the-art and present strong baselines for sarcasm detection based on BERT pre-trained language models. We further improve our BERT models by fine-tuning them on related intermediate tasks before fine-tuning them on our target task. Specifically, relying on the correlation between sarcasm and (implied negative) sentiment and emotions, we explore a transfer learning framework that uses sentiment classification and emotion detection as individual intermediate tasks to infuse knowledge into the target task of sarcasm detection. Experimental results on three datasets that have different characteristics show that the BERT-based models outperform many previous models. |
| first_indexed | 2024-03-09T20:30:42Z |
| format | Article |
| id | doaj.art-ff3043ba4f57440fa0a883b93956d8f3 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T20:30:42Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-ff3043ba4f57440fa0a883b93956d8f32023-11-23T23:24:32ZengMDPI AGMathematics2227-73902022-03-0110584410.3390/math10050844Intermediate-Task Transfer Learning with BERT for Sarcasm DetectionEdoardo Savini0Cornelia Caragea1Department of Computer Science, University of Illinois at Chicago, Chicago, IL 60607, USADepartment of Computer Science, University of Illinois at Chicago, Chicago, IL 60607, USASarcasm detection plays an important role in natural language processing as it can impact the performance of many applications, including sentiment analysis, opinion mining, and stance detection. Despite substantial progress on sarcasm detection, the research results are scattered across datasets and studies. In this paper, we survey the current state-of-the-art and present strong baselines for sarcasm detection based on BERT pre-trained language models. We further improve our BERT models by fine-tuning them on related intermediate tasks before fine-tuning them on our target task. Specifically, relying on the correlation between sarcasm and (implied negative) sentiment and emotions, we explore a transfer learning framework that uses sentiment classification and emotion detection as individual intermediate tasks to infuse knowledge into the target task of sarcasm detection. Experimental results on three datasets that have different characteristics show that the BERT-based models outperform many previous models.https://www.mdpi.com/2227-7390/10/5/844sarcasm detectionintermediate-task transfer learningemotion-enriched sarcasm detection |
| spellingShingle | Edoardo Savini Cornelia Caragea Intermediate-Task Transfer Learning with BERT for Sarcasm Detection Mathematics sarcasm detection intermediate-task transfer learning emotion-enriched sarcasm detection |
| title | Intermediate-Task Transfer Learning with BERT for Sarcasm Detection |
| title_full | Intermediate-Task Transfer Learning with BERT for Sarcasm Detection |
| title_fullStr | Intermediate-Task Transfer Learning with BERT for Sarcasm Detection |
| title_full_unstemmed | Intermediate-Task Transfer Learning with BERT for Sarcasm Detection |
| title_short | Intermediate-Task Transfer Learning with BERT for Sarcasm Detection |
| title_sort | intermediate task transfer learning with bert for sarcasm detection |
| topic | sarcasm detection intermediate-task transfer learning emotion-enriched sarcasm detection |
| url | https://www.mdpi.com/2227-7390/10/5/844 |
| work_keys_str_mv | AT edoardosavini intermediatetasktransferlearningwithbertforsarcasmdetection AT corneliacaragea intermediatetasktransferlearningwithbertforsarcasmdetection |