BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study
Educational institutions typically gather feedback from beneficiaries through formal surveys. Offering open-ended questions allows students to express their opinions about matters that may not have been measured directly in closed-ended questions. However, responses to open-ended questions are typic...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10376061/ |
| _version_ | 1797361465721618432 |
|---|---|
| author | Khloud A. Alshaikh Omaima A. Almatrafi Yoosef B. Abushark |
| author_facet | Khloud A. Alshaikh Omaima A. Almatrafi Yoosef B. Abushark |
| author_sort | Khloud A. Alshaikh |
| collection | DOAJ |
| description | Educational institutions typically gather feedback from beneficiaries through formal surveys. Offering open-ended questions allows students to express their opinions about matters that may not have been measured directly in closed-ended questions. However, responses to open-ended questions are typically overlooked due to the time and effort required. Aspect-based sentiment analysis is used to automate the process of extracting fine-grained information from texts. This study aims to 1) examine the performance of different BERT-based models for aspect term extraction for Arabic text sourced from educational institution surveys; 2) develop a system that automates the ABSA process in a way that will automatically label survey responses. An end-to-end system was developed as a case study to extract aspect terms, identify their polarity, map extracted aspects to their respective categories, and aggregate category polarity. To accomplish this, the models were evaluated using an in-house dataset. The result showed that FAST-LCF-ATEPC, a multilingual checkpoint, outperformed other models including AraBERT, MARBERT, and QARiB, in the aspect-term extraction task, with an F1 score of 0.58. Hence, it was used for aspect-term polarity classification, showing an F1 score of 0.86. Mapping aspects to their respective categories using a predefined list yielded an average F1 score of 0.98. Furthermore, the polarities of the categories were aggregated to summarize the overall polarity for each category. The developed system can support Arabic educational institutions in harnessing valuable information in responses to open-ended survey questions, allowing decision-makers to better allocate resources, and improve facilities, services, and students’ learning experiences. |
| first_indexed | 2024-03-08T15:55:05Z |
| format | Article |
| id | doaj.art-e62a71c1aa334b72aae78036940454f3 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T15:55:05Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e62a71c1aa334b72aae78036940454f32024-01-09T00:04:35ZengIEEEIEEE Access2169-35362024-01-01122288230210.1109/ACCESS.2023.334834210376061BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case StudyKhloud A. Alshaikh0https://orcid.org/0000-0002-7809-5308Omaima A. Almatrafi1https://orcid.org/0000-0003-2105-2275Yoosef B. Abushark2Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaEducational institutions typically gather feedback from beneficiaries through formal surveys. Offering open-ended questions allows students to express their opinions about matters that may not have been measured directly in closed-ended questions. However, responses to open-ended questions are typically overlooked due to the time and effort required. Aspect-based sentiment analysis is used to automate the process of extracting fine-grained information from texts. This study aims to 1) examine the performance of different BERT-based models for aspect term extraction for Arabic text sourced from educational institution surveys; 2) develop a system that automates the ABSA process in a way that will automatically label survey responses. An end-to-end system was developed as a case study to extract aspect terms, identify their polarity, map extracted aspects to their respective categories, and aggregate category polarity. To accomplish this, the models were evaluated using an in-house dataset. The result showed that FAST-LCF-ATEPC, a multilingual checkpoint, outperformed other models including AraBERT, MARBERT, and QARiB, in the aspect-term extraction task, with an F1 score of 0.58. Hence, it was used for aspect-term polarity classification, showing an F1 score of 0.86. Mapping aspects to their respective categories using a predefined list yielded an average F1 score of 0.98. Furthermore, the polarities of the categories were aggregated to summarize the overall polarity for each category. The developed system can support Arabic educational institutions in harnessing valuable information in responses to open-ended survey questions, allowing decision-makers to better allocate resources, and improve facilities, services, and students’ learning experiences.https://ieeexplore.ieee.org/document/10376061/Arabic ABSAaspect extractionaspect-based sentiment analysisBERT-based modeleducationpolarity classification |
| spellingShingle | Khloud A. Alshaikh Omaima A. Almatrafi Yoosef B. Abushark BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study IEEE Access Arabic ABSA aspect extraction aspect-based sentiment analysis BERT-based model education polarity classification |
| title | BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study |
| title_full | BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study |
| title_fullStr | BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study |
| title_full_unstemmed | BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study |
| title_short | BERT-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic Open-Ended Survey Responses: A Case Study |
| title_sort | bert based model for aspect based sentiment analysis for analyzing arabic open ended survey responses a case study |
| topic | Arabic ABSA aspect extraction aspect-based sentiment analysis BERT-based model education polarity classification |
| url | https://ieeexplore.ieee.org/document/10376061/ |
| work_keys_str_mv | AT khloudaalshaikh bertbasedmodelforaspectbasedsentimentanalysisforanalyzingarabicopenendedsurveyresponsesacasestudy AT omaimaaalmatrafi bertbasedmodelforaspectbasedsentimentanalysisforanalyzingarabicopenendedsurveyresponsesacasestudy AT yoosefbabushark bertbasedmodelforaspectbasedsentimentanalysisforanalyzingarabicopenendedsurveyresponsesacasestudy |