CRAB: cross-environment agent benchmark for multimodal language model agents

The development of autonomous agents increasingly relies on Multimodal Language Models (MLMs) to perform tasks described in natural language with GUI environments, such as websites, desktop computers, or mobile phones. Existing benchmarks for MLM agents in interactive environments are limited by the...

Full description

Bibliographic Details
Main Authors: Xu, T, Chen, L, Wu, DJ, Chen, Y, Zhang, Z, Yao, X, Xie, Z, Liu, S, Qian, B, Yang, A, Jin, Z, Deng, J, Torr, P, Ghanem, B, Li, G
Format: Conference item
Language:English
Published: Association for the Advancement of Artificial Intellgence 2024