Multimodal learning with transformers: a survey

Multimodal learning with transformers: a survey

Transformer is a promising neural network learner, and has achieved great success in various machine learning tasks. Thanks to the recent prevalence of multimodal applications and Big Data, Transformer-based multimodal learning has become a hot topic in AI research. This paper presents a comprehensi...

书目详细资料
Main Authors:	Xu, P, Zhu, X, Clifton, DA
格式:	Journal article
语言:	English
出版:	IEEE 2023

相似书籍

Multimodal Federated Learning: A Survey
由: Liwei Che, et al.
出版: (2023-08-01)

A survey of multimodal machine learning
由: Peng CHEN, et al.
出版: (2020-05-01)

Deep Multimodal Representation Learning: A Survey
由: Wenzhong Guo, et al.
出版: (2019-01-01)

A medical multimodal large language model for future pandemics
由: Liu, F, et al.
出版: (2023)

Survey of Deep Learning Based Multimodal Emotion Recognition
由: ZHAO Xiaoming, YANG Yijiao, ZHANG Shiqing
出版: (2022-07-01)

Deep Learning for Obstructive Sleep Apnea Detection and Severity Assessment: A Multimodal Signals Fusion Multiscale Transformer Model
由: Zhang Y, et al.
出版: (2025-01-01)

A systematic survey on multimodal emotion recognition using learning algorithms
由: Naveed Ahmed, et al.
出版: (2023-02-01)

Simple and Effective Multimodal Learning Based on Pre-Trained Transformer Models
由: Kazuki Miyazawa, et al.
出版: (2022-01-01)

Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation
由: Sandro Pezzelle, et al.
出版: (2021-01-01)

Assessing severity of pediatric pneumonia using multimodal transformers with multi-task learning
由: Jing Li, et al.
出版: (2024-12-01)

An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification
由: Jiajian Zhu, et al.
出版: (2024-01-01)

A medical multimodal large language model for future pandemics
由: Fenglin Liu, et al.
出版: (2023-12-01)

MEDT: Using Multimodal Encoding-Decoding Network as in Transformer for Multimodal Sentiment Analysis
由: Qingfu Qi, et al.
出版: (2022-01-01)

ZeroNLG: aligning and autoencoding domains for zero-shot multimodal and multilingual natural language generation
由: Yang, B, et al.
出版: (2024)

Survey of Multimodal Medical Question Answering
由: Hilmi Demirhan, et al.
出版: (2023-12-01)

Survey of research on multimodal semantic communication
由: Zhijin QIN, et al.
出版: (2023-05-01)

Survey of Multimodal Data Fusion Research
由: ZHANG Hucheng, LI Leixiao, LIU Dongjiang
出版: (2024-10-01)

Survey of research on multimodal semantic communication
由: Zhijin QIN, et al.
出版: (2023-05-01)

A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
由: Hailun Lian, et al.
出版: (2023-10-01)

A Short Survey on Deep Learning for Multimodal Integration: Applications, Future Perspectives and Challenges
由: Giovanna Maria Dimitri
出版: (2022-11-01)

Multimodal Interaction, Interfaces, and Communication: A Survey
由: Elias Dritsas, et al.
出版: (2025-01-01)

MMSFormer: Multimodal Transformer for Material and Semantic Segmentation
由: Md Kaykobad Reza, et al.
出版: (2024-01-01)

HGTMFS: A Hypergraph Transformer Framework for Multimodal Summarization
由: Ming Lu, et al.
出版: (2024-10-01)

Probabilistic Patient Monitoring with Multivariate, Multimodal Extreme Value Theory
由: Hugueny, S, et al.
出版: (2011)

CMJRT: Cross-Modal Joint Representation Transformer for Multimodal Sentiment Analysis
由: Meng Xu, et al.
出版: (2022-01-01)

Multi-Label Multimodal Emotion Recognition With Transformer-Based Fusion and Emotion-Level Representation Learning
由: Hoai-Duy Le, et al.
出版: (2023-01-01)

Multimodality Medical Image Fusion Using Clustered Dictionary Learning in Non-Subsampled Shearlet Transform
由: Manoj Diwakar, et al.
出版: (2023-04-01)

A Survey on Multimodal Aspect-Based Sentiment Analysis
由: Hua Zhao, et al.
出版: (2024-01-01)

Reading Multimodal Texts for Learning – a Model for Cultivating Multimodal Literacy
由: Kristina Danielsson, et al.
出版: (2016-08-01)

MMSFT: Multilingual Multimodal Summarization by Fine-Tuning Transformers
由: Siginamsetty Phani, et al.
出版: (2024-01-01)

Augmenting Multimodal Content Representation with Transformers for Misinformation Detection
由: Jenq-Haur Wang, et al.
出版: (2024-10-01)

Multimodal transformer augmented fusion for speech emotion recognition
由: Yuanyuan Wang, et al.
出版: (2023-05-01)

Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
由: Lisa Anne Hendricks, et al.
出版: (2021-01-01)

Deep scaffold hopping with multimodal transformer neural networks
由: Shuangjia Zheng, et al.
出版: (2021-11-01)

A multimodal multidomain multilingual medical foundation model for zero shot clinical diagnosis
由: Liu, F, et al.
出版: (2025)

A knowledge-enhanced transform-based multimodal classifier for microbial keratitis identification
由: Jianfeng Wu, et al.
出版: (2023-06-01)

A Comprehensive Survey for Deep-Learning-Based Abnormality Detection in Smart Grids with Multimodal Image Data
由: Fangrong Zhou, et al.
出版: (2022-05-01)

Survey of transformers and towards ensemble learning using transformers for natural language processing
由: Hongzhi Zhang, et al.
出版: (2024-02-01)

Advanced multiple document summarization via iterative recursive transformer networks and multimodal transformer
由: Sunilkumar Ketineni, et al.
出版: (2024-12-01)

Sarcopenia as a Prognostic Factor and Multimodal Interventions in Breast Cancer
由: Du L, et al.
出版: (2024-12-01)