Rethinking visual prompting for multimodal large language models with external knowledge

Rethinking visual prompting for multimodal large language models with external knowledge

In recent years, multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets, enabling them to generally understand images well. However, the inherent difficulty in explicitly conveying fine-grained or spatially dense information in tex...

Полное описание

Библиографические подробности
Главные авторы:	Lin, Y, Li, Y, Chen, D, Xu, W, Clark, R, Torr, P, Yuan, L
Формат:	Internet publication
Язык:	English
Опубликовано:	2024

Схожие документы

Prompting Large Language Models with Knowledge-Injection for Knowledge-Based Visual Question Answering
по: Zhongjian Hu, и др.
Опубликовано: (2024-09-01)

Knowledge graph construction for heart failure using large language models with prompt engineering
по: Tianhan Xu, и др.
Опубликовано: (2024-07-01)

Prompt Optimization in Large Language Models
по: Antonio Sabbatella, и др.
Опубликовано: (2024-03-01)

CAT: enhancing multimodal large language model to answer questions in dynamic audio-visual scenarios
по: Ye, Q, и др.
Опубликовано: (2024)

Review of large vision models and visual prompt engineering
по: Jiaqi Wang, и др.
Опубликовано: (2023-11-01)

A unified prompt-based framework for few-shot multimodal language analysis
по: Xiaohan Zhang, и др.
Опубликовано: (2025-06-01)

Learning visual prompts for guiding the attention of vision transformers
по: Rezaei, R, и др.
Опубликовано: (2024)

REKP: Refined External Knowledge into Prompt-Tuning for Few-Shot Text Classification
по: Yuzhuo Dang, и др.
Опубликовано: (2023-11-01)

Improving language model predictions via prompts enriched with knowledge graphs
по: Brate, R, и др.
Опубликовано: (2023)

Aligning, autoencoding and prompting large language models for novel disease reporting
по: Liu, F, и др.
Опубликовано: (2025)

uCAP: an unsupervised prompting method for vision-language models
по: Nguyen, AT, и др.
Опубликовано: (2024)

Predictive Prompts with Joint Training of Large Language Models for Explainable Recommendation
по: Ching-Sheng Lin, и др.
Опубликовано: (2023-10-01)

Extracting Fruit Disease Knowledge from Research Papers Based on Large Language Models and Prompt Engineering
по: Yunqiao Fei, и др.
Опубликовано: (2025-01-01)

Balancing Privacy and Robustness in Prompt Learning for Large Language Models
по: Chiyu Shi, и др.
Опубликовано: (2024-10-01)

Response Generated by Large Language Models Depends on the Structure of the Prompt
по: Pradosh Kumar Sarangi, и др.
Опубликовано: (2024-07-01)

Prompt Engineering: Guiding the Way to Effective Large Language Models
по: Mohammad Aljanabi, и др.
Опубликовано: (2023-11-01)

An image is worth 1000 lies: adversarial transferability across prompts on vision-language models
по: Luo, H, и др.
Опубликовано: (2024)

A Brief Overview of Few-Shot Prompting in the Large Language Models
по: Vladlen Kulikov, и др.
Опубликовано: (2023-05-01)

Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine
по: Thomas Savage, и др.
Опубликовано: (2024-01-01)

The application of multimodal large language models in medicine
по: Jianing Qiu, и др.
Опубликовано: (2024-04-01)

Clinical prompt learning with frozen language models
по: Taylor, N, и др.
Опубликовано: (2023)

LLMR: Real-time Prompting of Interactive Worlds using Large Language Models
по: De La Torre, Fernanda, и др.
Опубликовано: (2024)

Large language model enhanced with prompt-based vanilla distillation for sentence embeddings
по: Wang, Minghao
Опубликовано: (2024)

Large multimodal models for visual reasoning
по: Duong, Ngoc Yen
Опубликовано: (2024)

Intelligent extraction of reservoir dispatching information integrating large language model and structured prompts
по: Yangrui Yang, и др.
Опубликовано: (2024-06-01)

A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models
по: Erik Derner, и др.
Опубликовано: (2024-01-01)

DetToolChain: a new prompting paradigm to unleash detection ability of MLLM
по: Wu, Y, и др.
Опубликовано: (2024)

Research and application of defense mechanism for prompt injection attack of large language model in financial industry
по: MOU Daen, и др.
Опубликовано: (2024-10-01)

A medical multimodal large language model for future pandemics
по: Liu, F, и др.
Опубликовано: (2023)

On the legal implications of Large Language Model answers: A prompt engineering approach and a view beyond by exploiting Knowledge Graphs
по: George Hannah, и др.
Опубликовано: (2025-01-01)

Rethinking Language
по: Gastor Mapunda, и др.
Опубликовано: (2024-09-01)

Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation
по: Cyril Chhun, и др.
Опубликовано: (2024-09-01)

Harnessing multimodal large language models for traffic knowledge graph generation and decision-making
по: Senyun Kuang, и др.
Опубликовано: (2024-12-01)

PromptSMILES: prompting for scaffold decoration and fragment linking in chemical language models
по: Morgan Thomas, и др.
Опубликовано: (2024-07-01)

The influence of knowledge visualization on externalizing tacit knowledge
по: Ahmad, Khairul Bariah, и др.
Опубликовано: (2011)

Rethinking of Coase Theorem: Externalities and Uncertainty
по: Evgeny A. Kuzmin, и др.
Опубликовано: (2015-10-01)

Rethinking of Coase Theorem: Externalities and Uncertainty
по: Evgeny A. Kuzmin, и др.
Опубликовано: (2015-10-01)

Rethinking of Coase Theorem: Externalities and Uncertainty
по: Evgeny A. Kuzmin, и др.
Опубликовано: (2015-10-01)

Rethinking of Coase Theorem: Externalities and Uncertainty
по: Evgeny A. Kuzmin, и др.
Опубликовано: (2015-12-01)

TEACHING ENGLISH AS A FOREIGN LANGUAGE: RETHINKING THE MULTIMODALITY AND COMMUNICATION SKILLS IN THE 21st CENTURY
по: Liudmyla Byrkun
Опубликовано: (2023-12-01)