Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer
In the realm of the five-category classification endeavor, there has been limited exploration of applied techniques for classifying Arabic text. These methods have primarily leaned on single-task learning, incorporating manually crafted features that lack robust sentence representations. Recently, t...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/24/4960 |
| _version_ | 1797380137425043456 |
|---|---|
| author | Laith H. Baniata Sangwoo Kang |
| author_facet | Laith H. Baniata Sangwoo Kang |
| author_sort | Laith H. Baniata |
| collection | DOAJ |
| description | In the realm of the five-category classification endeavor, there has been limited exploration of applied techniques for classifying Arabic text. These methods have primarily leaned on single-task learning, incorporating manually crafted features that lack robust sentence representations. Recently, the Transformer paradigm has emerged as a highly promising alternative. However, when these models are trained using single-task learning, they often face challenges in achieving outstanding performance and generating robust latent feature representations, especially when dealing with small datasets. This issue is particularly pronounced in the context of the Arabic dialect, which has a scarcity of available resources. Given these constraints, this study introduces an innovative approach to dissecting sentiment in Arabic text. This approach combines Inductive Transfer (INT) with the Transformer paradigm to augment the adaptability of the model and refine the representation of sentences. By employing self-attention SE-A and feed-forward sub-layers as a shared Transformer encoder for both the five-category and three-category Arabic text classification tasks, this proposed model adeptly discerns sentiment in Arabic dialect sentences. The empirical findings underscore the commendable performance of the proposed model, as demonstrated in assessments of the Hotel Arabic-Reviews Dataset, the Book Reviews Arabic Dataset, and the LARB dataset. |
| first_indexed | 2024-03-08T20:33:57Z |
| format | Article |
| id | doaj.art-84d3869face4433e80036fd3f28d23a0 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-08T20:33:57Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-84d3869face4433e80036fd3f28d23a02023-12-22T14:23:26ZengMDPI AGMathematics2227-73902023-12-011124496010.3390/math11244960Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive TransferLaith H. Baniata0Sangwoo Kang1School of Computing, Gachon University, Seongnam 13120, Republic of KoreaSchool of Computing, Gachon University, Seongnam 13120, Republic of KoreaIn the realm of the five-category classification endeavor, there has been limited exploration of applied techniques for classifying Arabic text. These methods have primarily leaned on single-task learning, incorporating manually crafted features that lack robust sentence representations. Recently, the Transformer paradigm has emerged as a highly promising alternative. However, when these models are trained using single-task learning, they often face challenges in achieving outstanding performance and generating robust latent feature representations, especially when dealing with small datasets. This issue is particularly pronounced in the context of the Arabic dialect, which has a scarcity of available resources. Given these constraints, this study introduces an innovative approach to dissecting sentiment in Arabic text. This approach combines Inductive Transfer (INT) with the Transformer paradigm to augment the adaptability of the model and refine the representation of sentences. By employing self-attention SE-A and feed-forward sub-layers as a shared Transformer encoder for both the five-category and three-category Arabic text classification tasks, this proposed model adeptly discerns sentiment in Arabic dialect sentences. The empirical findings underscore the commendable performance of the proposed model, as demonstrated in assessments of the Hotel Arabic-Reviews Dataset, the Book Reviews Arabic Dataset, and the LARB dataset.https://www.mdpi.com/2227-7390/11/24/4960TransformerInductive Transfertext classificationArabic dialectspositional encodingfive-polarity |
| spellingShingle | Laith H. Baniata Sangwoo Kang Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer Mathematics Transformer Inductive Transfer text classification Arabic dialects positional encoding five-polarity |
| title | Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer |
| title_full | Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer |
| title_fullStr | Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer |
| title_full_unstemmed | Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer |
| title_short | Transformer Text Classification Model for Arabic Dialects That Utilizes Inductive Transfer |
| title_sort | transformer text classification model for arabic dialects that utilizes inductive transfer |
| topic | Transformer Inductive Transfer text classification Arabic dialects positional encoding five-polarity |
| url | https://www.mdpi.com/2227-7390/11/24/4960 |
| work_keys_str_mv | AT laithhbaniata transformertextclassificationmodelforarabicdialectsthatutilizesinductivetransfer AT sangwookang transformertextclassificationmodelforarabicdialectsthatutilizesinductivetransfer |