Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion
One of the main tasks in the field of natural language processing (NLP) is the analysis of affective states (sentiment and emotional) based on written text, and attempts have improved dramatically in recent years. However, in studies on the Arabic language, machine learning or deep learning algorith...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/9/5609 |
| _version_ | 1797602978111160320 |
|---|---|
| author | Wafa Alshehri Nora Al-Twairesh Abdulrahman Alothaim |
| author_facet | Wafa Alshehri Nora Al-Twairesh Abdulrahman Alothaim |
| author_sort | Wafa Alshehri |
| collection | DOAJ |
| description | One of the main tasks in the field of natural language processing (NLP) is the analysis of affective states (sentiment and emotional) based on written text, and attempts have improved dramatically in recent years. However, in studies on the Arabic language, machine learning or deep learning algorithms were utilised to analyse sentiment and emotion more often than current pre-trained language models. Additionally, further pre-training the language model on specific tasks (i.e., within-task and cross-task adaptation) has not yet been investigated for Arabic in general, and for the sentiment and emotion task in particular. In this paper, we adapt a BERT-based Arabic pretrained language model for the sentiment and emotion tasks by further pre-training it on a sentiment and emotion corpus. Hence, we developed five new Arabic models: QST, QSR, QSRT, QE3, and QE6. Five sentiment and two emotion datasets spanning both small- and large-resource settings were used to evaluate the developed models. The adaptation approaches significantly enhanced the performance of seven Arabic sentiment and emotion datasets. The developed models showed excellent improvements over the sentiment and emotion datasets, which ranged from 0.15–4.71%. |
| first_indexed | 2024-03-11T04:23:23Z |
| format | Article |
| id | doaj.art-8e0a74bca3b1481eabf210ff8f05f552 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T04:23:23Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-8e0a74bca3b1481eabf210ff8f05f5522023-11-17T22:36:23ZengMDPI AGApplied Sciences2076-34172023-05-01139560910.3390/app13095609Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and EmotionWafa Alshehri0Nora Al-Twairesh1Abdulrahman Alothaim2STC’s Artificial Intelligence Chair, College of Computer and Information Sciences, King Saud University, Riyadh 11451, Saudi ArabiaSTC’s Artificial Intelligence Chair, College of Computer and Information Sciences, King Saud University, Riyadh 11451, Saudi ArabiaSTC’s Artificial Intelligence Chair, College of Computer and Information Sciences, King Saud University, Riyadh 11451, Saudi ArabiaOne of the main tasks in the field of natural language processing (NLP) is the analysis of affective states (sentiment and emotional) based on written text, and attempts have improved dramatically in recent years. However, in studies on the Arabic language, machine learning or deep learning algorithms were utilised to analyse sentiment and emotion more often than current pre-trained language models. Additionally, further pre-training the language model on specific tasks (i.e., within-task and cross-task adaptation) has not yet been investigated for Arabic in general, and for the sentiment and emotion task in particular. In this paper, we adapt a BERT-based Arabic pretrained language model for the sentiment and emotion tasks by further pre-training it on a sentiment and emotion corpus. Hence, we developed five new Arabic models: QST, QSR, QSRT, QE3, and QE6. Five sentiment and two emotion datasets spanning both small- and large-resource settings were used to evaluate the developed models. The adaptation approaches significantly enhanced the performance of seven Arabic sentiment and emotion datasets. The developed models showed excellent improvements over the sentiment and emotion datasets, which ranged from 0.15–4.71%.https://www.mdpi.com/2076-3417/13/9/5609sentiment analysisemotion detectionpretrained language modelsmodel adaptationtask-adaptation approach |
| spellingShingle | Wafa Alshehri Nora Al-Twairesh Abdulrahman Alothaim Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion Applied Sciences sentiment analysis emotion detection pretrained language models model adaptation task-adaptation approach |
| title | Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion |
| title_full | Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion |
| title_fullStr | Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion |
| title_full_unstemmed | Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion |
| title_short | Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion |
| title_sort | affect analysis in arabic text further pre training language models for sentiment and emotion |
| topic | sentiment analysis emotion detection pretrained language models model adaptation task-adaptation approach |
| url | https://www.mdpi.com/2076-3417/13/9/5609 |
| work_keys_str_mv | AT wafaalshehri affectanalysisinarabictextfurtherpretraininglanguagemodelsforsentimentandemotion AT noraaltwairesh affectanalysisinarabictextfurtherpretraininglanguagemodelsforsentimentandemotion AT abdulrahmanalothaim affectanalysisinarabictextfurtherpretraininglanguagemodelsforsentimentandemotion |