Training Human-AI Teams
AI systems are augmenting humans' capabilities in settings such as healthcare and programming, forming human-AI teams. To enable more accurate and timely decisions, we need to optimize the performance of the human-AI team directly. In this thesis, we utilize a mathematical framing of the human-...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/155508 |
| _version_ | 1826213788533129216 |
|---|---|
| author | Mozannar, Hussein |
| author2 | Sontag, David |
| author_facet | Sontag, David Mozannar, Hussein |
| author_sort | Mozannar, Hussein |
| collection | MIT |
| description | AI systems are augmenting humans' capabilities in settings such as healthcare and programming, forming human-AI teams. To enable more accurate and timely decisions, we need to optimize the performance of the human-AI team directly. In this thesis, we utilize a mathematical framing of the human-AI team and propose a set of methods that optimize the AI, the human, and the interface in which they communicate to enable better team performance. We first show how to provably train AI classifiers that complement humans and can defer the decision to humans when it is best to do so. However, in specific settings, AI cannot autonomously make decisions and thus only provides advice to humans. In that case, we build onboarding procedures that train humans to have an accurate mental model of the AI to enable appropriate reliance. Finally, we study how humans interact with large language models (LLMs) to write code. To understand current inefficiencies, we developed a taxonomy to categorize programmers' interactions with the LLM. Motivated by insight from the taxonomy, we leverage human feedback to know when to best display LLM suggestions. |
| first_indexed | 2024-09-23T15:54:51Z |
| format | Thesis |
| id | mit-1721.1/155508 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T15:54:51Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1555082024-07-09T03:05:57Z Training Human-AI Teams Mozannar, Hussein Sontag, David Satyanarayan, Arvind Glassman, Elena Horvitz, Eric Massachusetts Institute of Technology. Institute for Data, Systems, and Society AI systems are augmenting humans' capabilities in settings such as healthcare and programming, forming human-AI teams. To enable more accurate and timely decisions, we need to optimize the performance of the human-AI team directly. In this thesis, we utilize a mathematical framing of the human-AI team and propose a set of methods that optimize the AI, the human, and the interface in which they communicate to enable better team performance. We first show how to provably train AI classifiers that complement humans and can defer the decision to humans when it is best to do so. However, in specific settings, AI cannot autonomously make decisions and thus only provides advice to humans. In that case, we build onboarding procedures that train humans to have an accurate mental model of the AI to enable appropriate reliance. Finally, we study how humans interact with large language models (LLMs) to write code. To understand current inefficiencies, we developed a taxonomy to categorize programmers' interactions with the LLM. Motivated by insight from the taxonomy, we leverage human feedback to know when to best display LLM suggestions. Ph.D. 2024-07-08T18:55:54Z 2024-07-08T18:55:54Z 2024-05 2024-06-03T20:46:12.190Z Thesis https://hdl.handle.net/1721.1/155508 Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Mozannar, Hussein Training Human-AI Teams |
| title | Training Human-AI Teams |
| title_full | Training Human-AI Teams |
| title_fullStr | Training Human-AI Teams |
| title_full_unstemmed | Training Human-AI Teams |
| title_short | Training Human-AI Teams |
| title_sort | training human ai teams |
| url | https://hdl.handle.net/1721.1/155508 |
| work_keys_str_mv | AT mozannarhussein traininghumanaiteams |