No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages
Research in vision and language has made considerable progress thanks to benchmarks such as COCO. COCO captions focused on unambiguous facts in English; ArtEmis introduced subjective emotions and ArtELingo introduced some multilinguality (Chinese and Arabic). However we believe there should be more...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Association for Computational Linguistics
2024
|
| _version_ | 1824458778230128640 |
|---|---|
| author | Mohamed, Y Li, R Ahmad, IS Haydarov, K Torr, P Church, KW Elhoseiny, M |
| author_facet | Mohamed, Y Li, R Ahmad, IS Haydarov, K Torr, P Church, KW Elhoseiny, M |
| author_sort | Mohamed, Y |
| collection | OXFORD |
| description | Research in vision and language has made considerable progress thanks to benchmarks such as COCO. COCO captions focused on unambiguous facts in English; ArtEmis introduced subjective emotions and ArtELingo introduced some multilinguality (Chinese and Arabic). However we believe there should be more multilinguality. Hence, we present ArtELingo-28, a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations (140 annotations per image). Traditionally, vision research focused on unambiguous class labels, whereas ArtELingo-28 emphasizes diversity of opinions over languages and cultures. The challenge is to build machine learning systems that assign emotional captions to images. Baseline results will be presented for three novel conditions: Zero-Shot, Few-Shot and One-vs-All Zero-Shot. We find that cross-lingual transfer is more successful for culturally-related languages. Data and code will be made publicly available. |
| first_indexed | 2025-02-19T04:31:18Z |
| format | Conference item |
| id | oxford-uuid:3571efc7-6065-474a-97f5-95113092bf13 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2025-02-19T04:31:18Z |
| publishDate | 2024 |
| publisher | Association for Computational Linguistics |
| record_format | dspace |
| spelling | oxford-uuid:3571efc7-6065-474a-97f5-95113092bf132025-01-08T12:42:47ZNo culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languagesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:3571efc7-6065-474a-97f5-95113092bf13EnglishSymplectic ElementsAssociation for Computational Linguistics2024Mohamed, YLi, RAhmad, ISHaydarov, KTorr, PChurch, KWElhoseiny, MResearch in vision and language has made considerable progress thanks to benchmarks such as COCO. COCO captions focused on unambiguous facts in English; ArtEmis introduced subjective emotions and ArtELingo introduced some multilinguality (Chinese and Arabic). However we believe there should be more multilinguality. Hence, we present ArtELingo-28, a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations (140 annotations per image). Traditionally, vision research focused on unambiguous class labels, whereas ArtELingo-28 emphasizes diversity of opinions over languages and cultures. The challenge is to build machine learning systems that assign emotional captions to images. Baseline results will be presented for three novel conditions: Zero-Shot, Few-Shot and One-vs-All Zero-Shot. We find that cross-lingual transfer is more successful for culturally-related languages. Data and code will be made publicly available. |
| spellingShingle | Mohamed, Y Li, R Ahmad, IS Haydarov, K Torr, P Church, KW Elhoseiny, M No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages |
| title | No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages |
| title_full | No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages |
| title_fullStr | No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages |
| title_full_unstemmed | No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages |
| title_short | No culture left behind: ArtELingo-28, a benchmark of WikiArt with captions in 28 languages |
| title_sort | no culture left behind artelingo 28 a benchmark of wikiart with captions in 28 languages |
| work_keys_str_mv | AT mohamedy nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages AT lir nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages AT ahmadis nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages AT haydarovk nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages AT torrp nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages AT churchkw nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages AT elhoseinym nocultureleftbehindartelingo28abenchmarkofwikiartwithcaptionsin28languages |