Self-adaptive attention fusion for multimodal aspect-based sentiment analysis
Multimodal aspect term extraction (MATE) and multimodal aspect-oriented sentiment classification (MASC) are two crucial subtasks in multimodal sentiment analysis. The use of pretrained generative models has attracted increasing attention in aspect-based sentiment analysis (ABSA). However, the inhere...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIMS Press
2024-01-01
|
| Series: | Mathematical Biosciences and Engineering |
| Subjects: | |
| Online Access: | https://www.aimspress.com/article/doi/10.3934/mbe.2024056?viewType=HTML |
| _version_ | 1797326279357235200 |
|---|---|
| author | Ziyue Wang Junjun Guo |
| author_facet | Ziyue Wang Junjun Guo |
| author_sort | Ziyue Wang |
| collection | DOAJ |
| description | Multimodal aspect term extraction (MATE) and multimodal aspect-oriented sentiment classification (MASC) are two crucial subtasks in multimodal sentiment analysis. The use of pretrained generative models has attracted increasing attention in aspect-based sentiment analysis (ABSA). However, the inherent semantic gap between textual and visual modalities poses a challenge in transferring text-based generative pretraining models to image-text multimodal sentiment analysis tasks. To tackle this issue, this paper proposes a self-adaptive cross-modal attention fusion architecture for joint multimodal aspect-based sentiment analysis (JMABSA), which is a generative model based on an image-text selective fusion mechanism that aims to bridge the semantic gap between text and image representations and adaptively transfer a textual-based pretraining model to the multimodal JMASA task. We conducted extensive experiments on two benchmark datasets, and the experimental results show that our model significantly outperforms other state of the art approaches by a significant margin. |
| first_indexed | 2024-03-08T06:22:00Z |
| format | Article |
| id | doaj.art-15ad0ac9c87a467a8d23c3630dd5c950 |
| institution | Directory Open Access Journal |
| issn | 1551-0018 |
| language | English |
| last_indexed | 2024-03-08T06:22:00Z |
| publishDate | 2024-01-01 |
| publisher | AIMS Press |
| record_format | Article |
| series | Mathematical Biosciences and Engineering |
| spelling | doaj.art-15ad0ac9c87a467a8d23c3630dd5c9502024-02-04T01:24:26ZengAIMS PressMathematical Biosciences and Engineering1551-00182024-01-012111305132010.3934/mbe.2024056Self-adaptive attention fusion for multimodal aspect-based sentiment analysisZiyue Wang0Junjun Guo 11. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China 2. Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming 650500, China1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China 2. Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming 650500, ChinaMultimodal aspect term extraction (MATE) and multimodal aspect-oriented sentiment classification (MASC) are two crucial subtasks in multimodal sentiment analysis. The use of pretrained generative models has attracted increasing attention in aspect-based sentiment analysis (ABSA). However, the inherent semantic gap between textual and visual modalities poses a challenge in transferring text-based generative pretraining models to image-text multimodal sentiment analysis tasks. To tackle this issue, this paper proposes a self-adaptive cross-modal attention fusion architecture for joint multimodal aspect-based sentiment analysis (JMABSA), which is a generative model based on an image-text selective fusion mechanism that aims to bridge the semantic gap between text and image representations and adaptively transfer a textual-based pretraining model to the multimodal JMASA task. We conducted extensive experiments on two benchmark datasets, and the experimental results show that our model significantly outperforms other state of the art approaches by a significant margin.https://www.aimspress.com/article/doi/10.3934/mbe.2024056?viewType=HTMLnatural language processingsentiment analysisjoint multimodal aspect-based sentiment analysismultimodal fusionself-adaptive fusion |
| spellingShingle | Ziyue Wang Junjun Guo Self-adaptive attention fusion for multimodal aspect-based sentiment analysis Mathematical Biosciences and Engineering natural language processing sentiment analysis joint multimodal aspect-based sentiment analysis multimodal fusion self-adaptive fusion |
| title | Self-adaptive attention fusion for multimodal aspect-based sentiment analysis |
| title_full | Self-adaptive attention fusion for multimodal aspect-based sentiment analysis |
| title_fullStr | Self-adaptive attention fusion for multimodal aspect-based sentiment analysis |
| title_full_unstemmed | Self-adaptive attention fusion for multimodal aspect-based sentiment analysis |
| title_short | Self-adaptive attention fusion for multimodal aspect-based sentiment analysis |
| title_sort | self adaptive attention fusion for multimodal aspect based sentiment analysis |
| topic | natural language processing sentiment analysis joint multimodal aspect-based sentiment analysis multimodal fusion self-adaptive fusion |
| url | https://www.aimspress.com/article/doi/10.3934/mbe.2024056?viewType=HTML |
| work_keys_str_mv | AT ziyuewang selfadaptiveattentionfusionformultimodalaspectbasedsentimentanalysis AT junjunguo selfadaptiveattentionfusionformultimodalaspectbasedsentimentanalysis |