Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing
In recent years, computational models trained to do object recognition have become increasingly capable. Models have demonstrated significant improvements and have achieved saturated performance on many standard image classification benchmarks sparking discussion of whether these models have achieve...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/156943 |
| _version_ | 1824458223660302336 |
|---|---|
| author | Cummings, Jesse E. |
| author2 | Katz, Boris |
| author_facet | Katz, Boris Cummings, Jesse E. |
| author_sort | Cummings, Jesse E. |
| collection | MIT |
| description | In recent years, computational models trained to do object recognition have become increasingly capable. Models have demonstrated significant improvements and have achieved saturated performance on many standard image classification benchmarks sparking discussion of whether these models have achieved parity with human object recognition ability and whether we can consider this problem solved. However, these models continue to fail in real-world applications and in un-human-like ways creating a disparity between the performance that benchmarks report and the performance that users experience. In this thesis, we investigate why standard datasets are misaligned with real-world performance by exploring image recognition difficulty as defined by human psychophysics. Using behavioral experiments with humans, we are able to identify images that humans struggle to recognize and investigate the prevalence of these images in datasets and their effect on model performance. To shed light on how humans are able to recognize these images, we conduct preliminary analysis with neuroimaging to take the first steps at identifying the neural signature of image difficulty. |
| first_indexed | 2025-02-19T04:22:29Z |
| format | Thesis |
| id | mit-1721.1/156943 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2025-02-19T04:22:29Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1569432024-09-25T03:30:11Z Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing Cummings, Jesse E. Katz, Boris Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science In recent years, computational models trained to do object recognition have become increasingly capable. Models have demonstrated significant improvements and have achieved saturated performance on many standard image classification benchmarks sparking discussion of whether these models have achieved parity with human object recognition ability and whether we can consider this problem solved. However, these models continue to fail in real-world applications and in un-human-like ways creating a disparity between the performance that benchmarks report and the performance that users experience. In this thesis, we investigate why standard datasets are misaligned with real-world performance by exploring image recognition difficulty as defined by human psychophysics. Using behavioral experiments with humans, we are able to identify images that humans struggle to recognize and investigate the prevalence of these images in datasets and their effect on model performance. To shed light on how humans are able to recognize these images, we conduct preliminary analysis with neuroimaging to take the first steps at identifying the neural signature of image difficulty. M.Eng. 2024-09-24T18:22:03Z 2024-09-24T18:22:03Z 2024-05 2024-07-11T14:37:33.668Z Thesis https://hdl.handle.net/1721.1/156943 Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Cummings, Jesse E. Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing |
| title | Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing |
| title_full | Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing |
| title_fullStr | Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing |
| title_full_unstemmed | Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing |
| title_short | Characterizing Image Recognition Difficulty in Artificial and Biological Visual Processing |
| title_sort | characterizing image recognition difficulty in artificial and biological visual processing |
| url | https://hdl.handle.net/1721.1/156943 |
| work_keys_str_mv | AT cummingsjessee characterizingimagerecognitiondifficultyinartificialandbiologicalvisualprocessing |