Rethinking visual prompting for multimodal large language models with external knowledge

Rethinking visual prompting for multimodal large language models with external knowledge

In recent years, multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets, enabling them to generally understand images well. However, the inherent difficulty in explicitly conveying fine-grained or spatially dense information in tex...

Cur síos iomlán

Sonraí bibleagrafaíochta
Príomhchruthaitheoirí:	Lin, Y, Li, Y, Chen, D, Xu, W, Clark, R, Torr, P, Yuan, L
Formáid:	Internet publication
Teanga:	English
Foilsithe / Cruthaithe:	2024

Míreanna comhchosúla

Prompting Large Language Models with Knowledge-Injection for Knowledge-Based Visual Question Answering
de réir: Zhongjian Hu, et al.
Foilsithe / Cruthaithe: (2024-09-01)

Knowledge graph construction for heart failure using large language models with prompt engineering
de réir: Tianhan Xu, et al.
Foilsithe / Cruthaithe: (2024-07-01)

Prompt Optimization in Large Language Models
de réir: Antonio Sabbatella, et al.
Foilsithe / Cruthaithe: (2024-03-01)

CAT: enhancing multimodal large language model to answer questions in dynamic audio-visual scenarios
de réir: Ye, Q, et al.
Foilsithe / Cruthaithe: (2024)

Review of large vision models and visual prompt engineering
de réir: Jiaqi Wang, et al.
Foilsithe / Cruthaithe: (2023-11-01)

A unified prompt-based framework for few-shot multimodal language analysis
de réir: Xiaohan Zhang, et al.
Foilsithe / Cruthaithe: (2025-06-01)

Learning visual prompts for guiding the attention of vision transformers
de réir: Rezaei, R, et al.
Foilsithe / Cruthaithe: (2024)

REKP: Refined External Knowledge into Prompt-Tuning for Few-Shot Text Classification
de réir: Yuzhuo Dang, et al.
Foilsithe / Cruthaithe: (2023-11-01)

Improving language model predictions via prompts enriched with knowledge graphs
de réir: Brate, R, et al.
Foilsithe / Cruthaithe: (2023)

Aligning, autoencoding and prompting large language models for novel disease reporting
de réir: Liu, F, et al.
Foilsithe / Cruthaithe: (2025)

uCAP: an unsupervised prompting method for vision-language models
de réir: Nguyen, AT, et al.
Foilsithe / Cruthaithe: (2024)

Predictive Prompts with Joint Training of Large Language Models for Explainable Recommendation
de réir: Ching-Sheng Lin, et al.
Foilsithe / Cruthaithe: (2023-10-01)

Extracting Fruit Disease Knowledge from Research Papers Based on Large Language Models and Prompt Engineering
de réir: Yunqiao Fei, et al.
Foilsithe / Cruthaithe: (2025-01-01)

Balancing Privacy and Robustness in Prompt Learning for Large Language Models
de réir: Chiyu Shi, et al.
Foilsithe / Cruthaithe: (2024-10-01)

Response Generated by Large Language Models Depends on the Structure of the Prompt
de réir: Pradosh Kumar Sarangi, et al.
Foilsithe / Cruthaithe: (2024-07-01)

Prompt Engineering: Guiding the Way to Effective Large Language Models
de réir: Mohammad Aljanabi, et al.
Foilsithe / Cruthaithe: (2023-11-01)

An image is worth 1000 lies: adversarial transferability across prompts on vision-language models
de réir: Luo, H, et al.
Foilsithe / Cruthaithe: (2024)

A Brief Overview of Few-Shot Prompting in the Large Language Models
de réir: Vladlen Kulikov, et al.
Foilsithe / Cruthaithe: (2023-05-01)

Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine
de réir: Thomas Savage, et al.
Foilsithe / Cruthaithe: (2024-01-01)

The application of multimodal large language models in medicine
de réir: Jianing Qiu, et al.
Foilsithe / Cruthaithe: (2024-04-01)

Clinical prompt learning with frozen language models
de réir: Taylor, N, et al.
Foilsithe / Cruthaithe: (2023)

LLMR: Real-time Prompting of Interactive Worlds using Large Language Models
de réir: De La Torre, Fernanda, et al.
Foilsithe / Cruthaithe: (2024)

Large language model enhanced with prompt-based vanilla distillation for sentence embeddings
de réir: Wang, Minghao
Foilsithe / Cruthaithe: (2024)

Large multimodal models for visual reasoning
de réir: Duong, Ngoc Yen
Foilsithe / Cruthaithe: (2024)

Intelligent extraction of reservoir dispatching information integrating large language model and structured prompts
de réir: Yangrui Yang, et al.
Foilsithe / Cruthaithe: (2024-06-01)

A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models
de réir: Erik Derner, et al.
Foilsithe / Cruthaithe: (2024-01-01)

DetToolChain: a new prompting paradigm to unleash detection ability of MLLM
de réir: Wu, Y, et al.
Foilsithe / Cruthaithe: (2024)

Research and application of defense mechanism for prompt injection attack of large language model in financial industry
de réir: MOU Daen, et al.
Foilsithe / Cruthaithe: (2024-10-01)

A medical multimodal large language model for future pandemics
de réir: Liu, F, et al.
Foilsithe / Cruthaithe: (2023)

On the legal implications of Large Language Model answers: A prompt engineering approach and a view beyond by exploiting Knowledge Graphs
de réir: George Hannah, et al.
Foilsithe / Cruthaithe: (2025-01-01)

Rethinking Language
de réir: Gastor Mapunda, et al.
Foilsithe / Cruthaithe: (2024-09-01)

Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation
de réir: Cyril Chhun, et al.
Foilsithe / Cruthaithe: (2024-09-01)

Harnessing multimodal large language models for traffic knowledge graph generation and decision-making
de réir: Senyun Kuang, et al.
Foilsithe / Cruthaithe: (2024-12-01)

PromptSMILES: prompting for scaffold decoration and fragment linking in chemical language models
de réir: Morgan Thomas, et al.
Foilsithe / Cruthaithe: (2024-07-01)

The influence of knowledge visualization on externalizing tacit knowledge
de réir: Ahmad, Khairul Bariah, et al.
Foilsithe / Cruthaithe: (2011)

Rethinking of Coase Theorem: Externalities and Uncertainty
de réir: Evgeny A. Kuzmin, et al.
Foilsithe / Cruthaithe: (2015-10-01)

Rethinking of Coase Theorem: Externalities and Uncertainty
de réir: Evgeny A. Kuzmin, et al.
Foilsithe / Cruthaithe: (2015-10-01)

Rethinking of Coase Theorem: Externalities and Uncertainty
de réir: Evgeny A. Kuzmin, et al.
Foilsithe / Cruthaithe: (2015-10-01)

Rethinking of Coase Theorem: Externalities and Uncertainty
de réir: Evgeny A. Kuzmin, et al.
Foilsithe / Cruthaithe: (2015-12-01)

TEACHING ENGLISH AS A FOREIGN LANGUAGE: RETHINKING THE MULTIMODALITY AND COMMUNICATION SKILLS IN THE 21st CENTURY
de réir: Liudmyla Byrkun
Foilsithe / Cruthaithe: (2023-12-01)