The Science and Art of Human and Artificial Intelligence Collaboration
While artificial intelligence (AI) appears to be surpassing the performance of human experts on a wide variety of games and real-world tasks, these algorithms are prone to systematic and surprising failures when deployed. In contrast to today’s state-of-the-art algorithms, humans are highly capable...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/152001 |
| _version_ | 1826188705580187648 |
|---|---|
| author | Groh, Matthew |
| author2 | Picard, Rosalind |
| author_facet | Picard, Rosalind Groh, Matthew |
| author_sort | Groh, Matthew |
| collection | MIT |
| description | While artificial intelligence (AI) appears to be surpassing the performance of human experts on a wide variety of games and real-world tasks, these algorithms are prone to systematic and surprising failures when deployed. In contrast to today’s state-of-the-art algorithms, humans are highly capable of adapting to new contexts. The different strengths and weaknesses of humans and AI motivate a guiding research question for the emerging field of human-AI collaboration: When, where, why, and how does the combination of human problem solving and AI systems lead to a hybrid system that surpasses (or fails to surpass) the performance of either humans or the machine alone? This dissertation addresses various dimensions of this guiding question by conducting large-scale, digital experiments across three distinct tasks and domains: deepfake detection, dermatology diagnosis, and Wordle. First, the experiments in deepfake detection examine the similarities and differences between human and machine vision in identifying visual manipulations of people’s faces in videos and identify important performance trade-offs between hybrid systems and human or AI only systems for deepfake detection. Second, the experiments in dermatology diagnosis reveal that non-visual information is often essential for diagnosing skin disease, diagnostic accuracy disparities across skin color exist in image-only store-and-forward teledermatology, and clinical decision support based on a fair deep learning system can significantly increase physicians’ diagnostic accuracy in this experimental setting. Third, the experiment on Wordle demonstrates that digitally mediated expressions of empathy can counteract the negative effect of anger on human creative problem solving. In addition to these digital experiments, this dissertation presents two algorithmic audits on clinical dermatology images to reveal where systematic errors arise in state-of-the-art algorithms, examines how context influences automated affect recognition, and proposes methods for more effectively incorporating context in applied machine learning. Together, these contributions provide empirical evidence for why human-AI collaborations succeed and fail across a variety of tasks and domains, insights into how to design human-AI collaborations more effectively, and a framework for when and where hybrid systems should rely on human problem solving. |
| first_indexed | 2024-09-23T08:03:46Z |
| format | Thesis |
| id | mit-1721.1/152001 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T08:03:46Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1520012023-09-01T03:59:04Z The Science and Art of Human and Artificial Intelligence Collaboration Groh, Matthew Picard, Rosalind Program in Media Arts and Sciences (Massachusetts Institute of Technology) While artificial intelligence (AI) appears to be surpassing the performance of human experts on a wide variety of games and real-world tasks, these algorithms are prone to systematic and surprising failures when deployed. In contrast to today’s state-of-the-art algorithms, humans are highly capable of adapting to new contexts. The different strengths and weaknesses of humans and AI motivate a guiding research question for the emerging field of human-AI collaboration: When, where, why, and how does the combination of human problem solving and AI systems lead to a hybrid system that surpasses (or fails to surpass) the performance of either humans or the machine alone? This dissertation addresses various dimensions of this guiding question by conducting large-scale, digital experiments across three distinct tasks and domains: deepfake detection, dermatology diagnosis, and Wordle. First, the experiments in deepfake detection examine the similarities and differences between human and machine vision in identifying visual manipulations of people’s faces in videos and identify important performance trade-offs between hybrid systems and human or AI only systems for deepfake detection. Second, the experiments in dermatology diagnosis reveal that non-visual information is often essential for diagnosing skin disease, diagnostic accuracy disparities across skin color exist in image-only store-and-forward teledermatology, and clinical decision support based on a fair deep learning system can significantly increase physicians’ diagnostic accuracy in this experimental setting. Third, the experiment on Wordle demonstrates that digitally mediated expressions of empathy can counteract the negative effect of anger on human creative problem solving. In addition to these digital experiments, this dissertation presents two algorithmic audits on clinical dermatology images to reveal where systematic errors arise in state-of-the-art algorithms, examines how context influences automated affect recognition, and proposes methods for more effectively incorporating context in applied machine learning. Together, these contributions provide empirical evidence for why human-AI collaborations succeed and fail across a variety of tasks and domains, insights into how to design human-AI collaborations more effectively, and a framework for when and where hybrid systems should rely on human problem solving. Ph.D. 2023-08-30T15:58:27Z 2023-08-30T15:58:27Z 2023-06 2023-08-16T20:34:12.141Z Thesis https://hdl.handle.net/1721.1/152001 0000-0002-9029-0157 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Groh, Matthew The Science and Art of Human and Artificial Intelligence Collaboration |
| title | The Science and Art of Human and Artificial Intelligence Collaboration |
| title_full | The Science and Art of Human and Artificial Intelligence Collaboration |
| title_fullStr | The Science and Art of Human and Artificial Intelligence Collaboration |
| title_full_unstemmed | The Science and Art of Human and Artificial Intelligence Collaboration |
| title_short | The Science and Art of Human and Artificial Intelligence Collaboration |
| title_sort | science and art of human and artificial intelligence collaboration |
| url | https://hdl.handle.net/1721.1/152001 |
| work_keys_str_mv | AT grohmatthew thescienceandartofhumanandartificialintelligencecollaboration AT grohmatthew scienceandartofhumanandartificialintelligencecollaboration |