Measuring and modeling the perception of natural and unconstrained gaze in humans and machines
Humans are remarkably adept at interpreting the gaze direction of other individuals in their surroundings. This skill is at the core of the ability to engage in joint visual attention, which is essential for establishing social interactions. How accurate are humans in determining the gaze direction...
| Main Authors: | , , , , |
|---|---|
| Format: | Technical Report |
| Language: | en_US |
| Published: |
Center for Brains, Minds and Machines (CBMM), arXiv
2016
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/105477 |
| _version_ | 1811083555939287040 |
|---|---|
| author | Harari, Daniel Gao, Tao Kanwisher, Nancy Tenenbaum, Joshua Ullman, Shimon |
| author_facet | Harari, Daniel Gao, Tao Kanwisher, Nancy Tenenbaum, Joshua Ullman, Shimon |
| author_sort | Harari, Daniel |
| collection | MIT |
| description | Humans are remarkably adept at interpreting the gaze direction of other individuals in their surroundings. This skill is at the core of the ability to engage in joint visual attention, which is essential for establishing social interactions. How accurate are humans in determining the gaze direction of others in lifelike scenes, when they can move their heads and eyes freely, and what are the sources of information for the underlying perceptual processes? These questions pose a challenge from both empirical and computational perspectives, due to the complexity of the visual input in real-life situations. Here we measure empirically human accuracy in perceiving the gaze direction of others in lifelike scenes, and study computationally the sources of information and representations underlying this cognitive capacity. We show that humans perform better in face-to-face conditions compared with recorded conditions, and that this advantage is not due to the availability of input dynamics. We further show that humans are still performing well when only the eyes-region is visible, rather than the whole face. We develop a computational model, which replicates the pattern of human performance, including the finding that the eyes-region contains on its own, the required information for estimating both head orientation and direction of gaze. Consistent with neurophysiological findings on task-specific face regions in the brain, the learned computational representations reproduce perceptual effects such as the Wollaston illusion, when trained to estimate direction of gaze, but not when trained to recognize objects or faces. |
| first_indexed | 2024-09-23T12:34:56Z |
| format | Technical Report |
| id | mit-1721.1/105477 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T12:34:56Z |
| publishDate | 2016 |
| publisher | Center for Brains, Minds and Machines (CBMM), arXiv |
| record_format | dspace |
| spelling | mit-1721.1/1054772019-04-11T09:51:54Z Measuring and modeling the perception of natural and unconstrained gaze in humans and machines Harari, Daniel Gao, Tao Kanwisher, Nancy Tenenbaum, Joshua Ullman, Shimon gaze direction social interaction human vision Humans are remarkably adept at interpreting the gaze direction of other individuals in their surroundings. This skill is at the core of the ability to engage in joint visual attention, which is essential for establishing social interactions. How accurate are humans in determining the gaze direction of others in lifelike scenes, when they can move their heads and eyes freely, and what are the sources of information for the underlying perceptual processes? These questions pose a challenge from both empirical and computational perspectives, due to the complexity of the visual input in real-life situations. Here we measure empirically human accuracy in perceiving the gaze direction of others in lifelike scenes, and study computationally the sources of information and representations underlying this cognitive capacity. We show that humans perform better in face-to-face conditions compared with recorded conditions, and that this advantage is not due to the availability of input dynamics. We further show that humans are still performing well when only the eyes-region is visible, rather than the whole face. We develop a computational model, which replicates the pattern of human performance, including the finding that the eyes-region contains on its own, the required information for estimating both head orientation and direction of gaze. Consistent with neurophysiological findings on task-specific face regions in the brain, the learned computational representations reproduce perceptual effects such as the Wollaston illusion, when trained to estimate direction of gaze, but not when trained to recognize objects or faces. This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF – 1231216. 2016-11-30T17:01:05Z 2016-11-30T17:01:05Z 2016-11-28 Technical Report Working Paper Other http://hdl.handle.net/1721.1/105477 arXiv:1611.09819 en_US CBMM Memo Series;059 Attribution-NonCommercial-ShareAlike 3.0 United States http://creativecommons.org/licenses/by-nc-sa/3.0/us/ application/pdf Center for Brains, Minds and Machines (CBMM), arXiv |
| spellingShingle | gaze direction social interaction human vision Harari, Daniel Gao, Tao Kanwisher, Nancy Tenenbaum, Joshua Ullman, Shimon Measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| title | Measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| title_full | Measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| title_fullStr | Measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| title_full_unstemmed | Measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| title_short | Measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| title_sort | measuring and modeling the perception of natural and unconstrained gaze in humans and machines |
| topic | gaze direction social interaction human vision |
| url | http://hdl.handle.net/1721.1/105477 |
| work_keys_str_mv | AT hararidaniel measuringandmodelingtheperceptionofnaturalandunconstrainedgazeinhumansandmachines AT gaotao measuringandmodelingtheperceptionofnaturalandunconstrainedgazeinhumansandmachines AT kanwishernancy measuringandmodelingtheperceptionofnaturalandunconstrainedgazeinhumansandmachines AT tenenbaumjoshua measuringandmodelingtheperceptionofnaturalandunconstrainedgazeinhumansandmachines AT ullmanshimon measuringandmodelingtheperceptionofnaturalandunconstrainedgazeinhumansandmachines |