MMRDF: An improved multitask multi‐modal rumor detection framework
Abstract As images are attractive and contained in plenty of posts, some studies show that combining multimodal information helps to represent posts effectively, while these approaches (1) simply concatenated unimodal features without considering inter‐modality relations, which failed to improve the...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-05-01
|
| Series: | Electronics Letters |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ell2.12811 |
| Summary: | Abstract As images are attractive and contained in plenty of posts, some studies show that combining multimodal information helps to represent posts effectively, while these approaches (1) simply concatenated unimodal features without considering inter‐modality relations, which failed to improve the prediction performance and generalization ability; (2) lacked careful consideration of the social network data composition structure, assuming that social network data was composed of image‐text pairs, cannot process the data composed of multi‐images, and cannot operate when one of the modalities is missing. To address these issues, a novel framework named multitask multimodal rumor detection framework (MMRDF) is proposed, comprising three sub‐networks, which is available to generate joint multimodal representation through merging different levels of features in multi‐modalities and can deal with all types of tweets (pure text, pure image, image‐text pair and text with multi‐images) flexibly. Since the framework does not take the data that causes extra time delay as inputs, such as propagation structures and response content, it can be applied as soon as the tweets are posted up and decrease the time delays of rumor debunking. Experiments on two real‐world datasets demonstrate the framework significantly outperforms the state‐of‐the‐art by the accuracy of 6.5% and 7.6%. |
|---|---|
| ISSN: | 0013-5194 1350-911X |