Multimodal Arabic Rumors Detection
Recently, the use of social media platforms has increased with ease of use and fast accessibility, making such platforms a place of rumor proliferation owing to the lack of posting constraints and content authentication. Therefore, there is a need to leverage artificial intelligence techniques to de...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10026837/ |
| _version_ | 1797804888636260352 |
|---|---|
| author | Rasha M. Albalawi Amani T. Jamal Alaa O. Khadidos Areej M. Alhothali |
| author_facet | Rasha M. Albalawi Amani T. Jamal Alaa O. Khadidos Areej M. Alhothali |
| author_sort | Rasha M. Albalawi |
| collection | DOAJ |
| description | Recently, the use of social media platforms has increased with ease of use and fast accessibility, making such platforms a place of rumor proliferation owing to the lack of posting constraints and content authentication. Therefore, there is a need to leverage artificial intelligence techniques to detect rumors on social media platforms to prevent their adverse effects on society and individuals. Most existing works that detect rumors in Arabic target the textual features of the tweet content. Nevertheless, tweets contain different types of content, such as (text, images, videos, and URLs), and the visual features of tweets play an essential role in rumor diffusion. This study proposes an Arabic rumor detection model to detect rumors on Twitter using textual and visual image features through two types of multimodal fusion: early and late fusion. In addition, we leveraged the transfer learning of the pre-trained language and vision models. Different experiments were conducted to select the best textual and visual feature extractors for building a multimodal model. MARBERTv2 was used as a textual feature extractor, whereas the ensemble of VGG-19 and ResNet50 was used as a visual feature extractor to build the multimodal model. Subsequently, the language and vision models of the single models were used as a baseline to compare their results with those of multimodal models. Finally, the experimental results demonstrate the effectiveness of textual features in rumor detection tasks compared to multimodal models. |
| first_indexed | 2024-03-13T05:43:49Z |
| format | Article |
| id | doaj.art-78428ff5ce1a4034ba15e8cdfac3028e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T05:43:49Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-78428ff5ce1a4034ba15e8cdfac3028e2023-06-13T20:37:41ZengIEEEIEEE Access2169-35362023-01-01119716973010.1109/ACCESS.2023.324037310026837Multimodal Arabic Rumors DetectionRasha M. Albalawi0https://orcid.org/0000-0002-3859-0611Amani T. Jamal1Alaa O. Khadidos2https://orcid.org/0000-0003-3297-3715Areej M. Alhothali3https://orcid.org/0000-0001-9727-0178Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi ArabiaRecently, the use of social media platforms has increased with ease of use and fast accessibility, making such platforms a place of rumor proliferation owing to the lack of posting constraints and content authentication. Therefore, there is a need to leverage artificial intelligence techniques to detect rumors on social media platforms to prevent their adverse effects on society and individuals. Most existing works that detect rumors in Arabic target the textual features of the tweet content. Nevertheless, tweets contain different types of content, such as (text, images, videos, and URLs), and the visual features of tweets play an essential role in rumor diffusion. This study proposes an Arabic rumor detection model to detect rumors on Twitter using textual and visual image features through two types of multimodal fusion: early and late fusion. In addition, we leveraged the transfer learning of the pre-trained language and vision models. Different experiments were conducted to select the best textual and visual feature extractors for building a multimodal model. MARBERTv2 was used as a textual feature extractor, whereas the ensemble of VGG-19 and ResNet50 was used as a visual feature extractor to build the multimodal model. Subsequently, the language and vision models of the single models were used as a baseline to compare their results with those of multimodal models. Finally, the experimental results demonstrate the effectiveness of textual features in rumor detection tasks compared to multimodal models.https://ieeexplore.ieee.org/document/10026837/Arabic NLPartificial intelligencedeep learningmultimodal fusionrumor detectiontransfer learning |
| spellingShingle | Rasha M. Albalawi Amani T. Jamal Alaa O. Khadidos Areej M. Alhothali Multimodal Arabic Rumors Detection IEEE Access Arabic NLP artificial intelligence deep learning multimodal fusion rumor detection transfer learning |
| title | Multimodal Arabic Rumors Detection |
| title_full | Multimodal Arabic Rumors Detection |
| title_fullStr | Multimodal Arabic Rumors Detection |
| title_full_unstemmed | Multimodal Arabic Rumors Detection |
| title_short | Multimodal Arabic Rumors Detection |
| title_sort | multimodal arabic rumors detection |
| topic | Arabic NLP artificial intelligence deep learning multimodal fusion rumor detection transfer learning |
| url | https://ieeexplore.ieee.org/document/10026837/ |
| work_keys_str_mv | AT rashamalbalawi multimodalarabicrumorsdetection AT amanitjamal multimodalarabicrumorsdetection AT alaaokhadidos multimodalarabicrumorsdetection AT areejmalhothali multimodalarabicrumorsdetection |