FigurA11y: AI Assistance for Writing Scientific Alt Text
IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces March 18–21, 2024, Greenville, SC, USA
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
ACM
2024
|
| Online Access: | https://hdl.handle.net/1721.1/154388 |
| _version_ | 1826203339063296000 |
|---|---|
| author | Singh, Nikhil Wang, Lucy Lu Bragg, Jonathan |
| author2 | Massachusetts Institute of Technology. Media Laboratory |
| author_facet | Massachusetts Institute of Technology. Media Laboratory Singh, Nikhil Wang, Lucy Lu Bragg, Jonathan |
| author_sort | Singh, Nikhil |
| collection | MIT |
| description | IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces March 18–21, 2024, Greenville, SC, USA |
| first_indexed | 2024-09-23T12:35:03Z |
| format | Article |
| id | mit-1721.1/154388 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2025-02-19T04:21:29Z |
| publishDate | 2024 |
| publisher | ACM |
| record_format | dspace |
| spelling | mit-1721.1/1543882025-01-04T05:14:24Z FigurA11y: AI Assistance for Writing Scientific Alt Text Singh, Nikhil Wang, Lucy Lu Bragg, Jonathan Massachusetts Institute of Technology. Media Laboratory IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces March 18–21, 2024, Greenville, SC, USA High-quality alt text is crucial for making scientific figures accessible to blind and low-vision readers. Crafting complete, accurate alt text is challenging even for domain experts, as published figures often depict complex visual information and readers have varied informational needs. These challenges, along with high diversity in figure types and domain-specific details, also limit the usefulness of fully automated approaches. Consequently, the prevalence of high-quality alt text is very low in scientific papers today. We investigate whether and how human-AI collaborative editing systems can help address the difficulty of writing high-quality alt text for complex scientific figures. We present FigurA11y, an interactive system that generates draft alt text and provides suggestions for author revisions using a pipeline driven by extracted figure and paper metadata. We test two versions, motivated by prior work on visual accessibility and writing support. The base Draft+Revise version provides authors with an automatically generated draft description to revise, along with extracted figure metadata and figure-specific alt text guidelines to support the revision process. The full Interactive Assistance version further adds contextualized suggestions: text snippets to iteratively produce descriptions, and hypothetical user questions with possible answers to reveal potential ambiguities and resolutions. In a study of authors (N=14), we found the system assisted them in efficiently producing descriptive alt text. Generated drafts and interface elements enabled authors to quickly initiate and edit detailed descriptions. Additionally, interactive suggestions from the full system prompted more iteration and highlighted aspects for authors to consider, resulting in greater deviation from the drafts without increased average cognitive load or manual effort. 2024-05-02T20:04:53Z 2024-05-02T20:04:53Z 2024-03-18 2024-05-01T07:47:31Z Article http://purl.org/eprint/type/JournalArticle 979-8-4007-0508-3 https://hdl.handle.net/1721.1/154388 Singh, Nikhil, Wang, Lucy Lu and Bragg, Jonathan. 2024. "FigurA11y: AI Assistance for Writing Scientific Alt Text." PUBLISHER_CC en 10.1145/3640543.3645212 Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ The author(s) application/pdf ACM Association for Computing Machinery |
| spellingShingle | Singh, Nikhil Wang, Lucy Lu Bragg, Jonathan FigurA11y: AI Assistance for Writing Scientific Alt Text |
| title | FigurA11y: AI Assistance for Writing Scientific Alt Text |
| title_full | FigurA11y: AI Assistance for Writing Scientific Alt Text |
| title_fullStr | FigurA11y: AI Assistance for Writing Scientific Alt Text |
| title_full_unstemmed | FigurA11y: AI Assistance for Writing Scientific Alt Text |
| title_short | FigurA11y: AI Assistance for Writing Scientific Alt Text |
| title_sort | figura11y ai assistance for writing scientific alt text |
| url | https://hdl.handle.net/1721.1/154388 |
| work_keys_str_mv | AT singhnikhil figura11yaiassistanceforwritingscientificalttext AT wanglucylu figura11yaiassistanceforwritingscientificalttext AT braggjonathan figura11yaiassistanceforwritingscientificalttext |