Where on the face do we look during phonemic restoration: An eye-tracking study
Face to face communication typically involves audio and visual components to the speech signal. To examine the effect of task demands on gaze patterns in response to a speaking face, adults participated in two eye-tracking experiments with an audiovisual (articulatory information from the mouth was...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-05-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1005186/full |
| _version_ | 1797820319101091840 |
|---|---|
| author | Alisa Baron Vanessa Harwood Daniel Kleinman Luca Campanelli Joseph Molski Nicole Landi Nicole Landi Julia Irwin Julia Irwin |
| author_facet | Alisa Baron Vanessa Harwood Daniel Kleinman Luca Campanelli Joseph Molski Nicole Landi Nicole Landi Julia Irwin Julia Irwin |
| author_sort | Alisa Baron |
| collection | DOAJ |
| description | Face to face communication typically involves audio and visual components to the speech signal. To examine the effect of task demands on gaze patterns in response to a speaking face, adults participated in two eye-tracking experiments with an audiovisual (articulatory information from the mouth was visible) and a pixelated condition (articulatory information was not visible). Further, task demands were manipulated by having listeners respond in a passive (no response) or an active (button press response) context. The active experiment required participants to discriminate between speech stimuli and was designed to mimic environmental situations which require one to use visual information to disambiguate the speaker’s message, simulating different listening conditions in real-world settings. Stimuli included a clear exemplar of the syllable /ba/ and a second exemplar in which the formant initial consonant was reduced creating an /a/−like consonant. Consistent with our hypothesis, results revealed that the greatest fixations to the mouth were present in the audiovisual active experiment and visual articulatory information led to a phonemic restoration effect for the /a/ speech token. In the pixelated condition, participants fixated on the eyes, and discrimination of the deviant token within the active experiment was significantly greater than the audiovisual condition. These results suggest that when required to disambiguate changes in speech, adults may look to the mouth for additional cues to support processing when it is available. |
| first_indexed | 2024-03-13T09:36:40Z |
| format | Article |
| id | doaj.art-ad4e5279ef534e7d884c1b44accde9ea |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-03-13T09:36:40Z |
| publishDate | 2023-05-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-ad4e5279ef534e7d884c1b44accde9ea2023-05-25T12:28:58ZengFrontiers Media S.A.Frontiers in Psychology1664-10782023-05-011410.3389/fpsyg.2023.10051861005186Where on the face do we look during phonemic restoration: An eye-tracking studyAlisa Baron0Vanessa Harwood1Daniel Kleinman2Luca Campanelli3Joseph Molski4Nicole Landi5Nicole Landi6Julia Irwin7Julia Irwin8Department of Communicative Disorders, University of Rhode Island, Kingston, RI, United StatesDepartment of Communicative Disorders, University of Rhode Island, Kingston, RI, United StatesHaskins Laboratories, New Haven, CT, United StatesDepartment of Communicative Disorders, The University of Alabama, Tuscaloosa, AL, United StatesDepartment of Communicative Disorders, University of Rhode Island, Kingston, RI, United StatesHaskins Laboratories, New Haven, CT, United StatesDepartment of Psychological Sciences, University of Connecticut, Storrs, CT, United StatesHaskins Laboratories, New Haven, CT, United StatesDepartment of Psychology, Southern Connecticut State University, New Haven, CT, United StatesFace to face communication typically involves audio and visual components to the speech signal. To examine the effect of task demands on gaze patterns in response to a speaking face, adults participated in two eye-tracking experiments with an audiovisual (articulatory information from the mouth was visible) and a pixelated condition (articulatory information was not visible). Further, task demands were manipulated by having listeners respond in a passive (no response) or an active (button press response) context. The active experiment required participants to discriminate between speech stimuli and was designed to mimic environmental situations which require one to use visual information to disambiguate the speaker’s message, simulating different listening conditions in real-world settings. Stimuli included a clear exemplar of the syllable /ba/ and a second exemplar in which the formant initial consonant was reduced creating an /a/−like consonant. Consistent with our hypothesis, results revealed that the greatest fixations to the mouth were present in the audiovisual active experiment and visual articulatory information led to a phonemic restoration effect for the /a/ speech token. In the pixelated condition, participants fixated on the eyes, and discrimination of the deviant token within the active experiment was significantly greater than the audiovisual condition. These results suggest that when required to disambiguate changes in speech, adults may look to the mouth for additional cues to support processing when it is available.https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1005186/fullaudiovisual integrationphonemic restorationspeech perceptioneye trackinggaze patterns |
| spellingShingle | Alisa Baron Vanessa Harwood Daniel Kleinman Luca Campanelli Joseph Molski Nicole Landi Nicole Landi Julia Irwin Julia Irwin Where on the face do we look during phonemic restoration: An eye-tracking study Frontiers in Psychology audiovisual integration phonemic restoration speech perception eye tracking gaze patterns |
| title | Where on the face do we look during phonemic restoration: An eye-tracking study |
| title_full | Where on the face do we look during phonemic restoration: An eye-tracking study |
| title_fullStr | Where on the face do we look during phonemic restoration: An eye-tracking study |
| title_full_unstemmed | Where on the face do we look during phonemic restoration: An eye-tracking study |
| title_short | Where on the face do we look during phonemic restoration: An eye-tracking study |
| title_sort | where on the face do we look during phonemic restoration an eye tracking study |
| topic | audiovisual integration phonemic restoration speech perception eye tracking gaze patterns |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1005186/full |
| work_keys_str_mv | AT alisabaron whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT vanessaharwood whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT danielkleinman whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT lucacampanelli whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT josephmolski whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT nicolelandi whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT nicolelandi whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT juliairwin whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy AT juliairwin whereonthefacedowelookduringphonemicrestorationaneyetrackingstudy |