Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts
It is well established that hand gestures affect comprehension and learning of semantic aspects of a foreign language (FL). However, much less is known about the role of hand gestures in lower-level language processes, such as perception of phonemes. To address this gap, we explored the role that me...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of California Press
2017-03-01
|
| Series: | Collabra: Psychology |
| Subjects: | |
| Online Access: | https://www.collabra.org/articles/76 |
| _version_ | 1819106779091435520 |
|---|---|
| author | Spencer Kelly April Bailey Yukari Hirata |
| author_facet | Spencer Kelly April Bailey Yukari Hirata |
| author_sort | Spencer Kelly |
| collection | DOAJ |
| description | It is well established that hand gestures affect comprehension and learning of semantic aspects of a foreign language (FL). However, much less is known about the role of hand gestures in lower-level language processes, such as perception of phonemes. To address this gap, we explored the role that metaphoric gestures play in perceiving FL speech sounds that varied on two dimensions: length and intonation. English speaking adults listened to Japanese length contrasts and sentence-final intonational distinctions in the context of congruent, incongruent and no gestures. For intonational contrasts, identification was more accurate for congruent gestures and less accurate for incongruent gestures relative to the baseline no gesture condition. However, for the length contrasts, there was no such clear and consistent pattern, and in fact, congruent gestures made speech processing more effortful. We conclude that metaphoric gestures help with some—but not all—novel speech sounds in a FL, suggesting that gesture and speech are phonemically integrated to differing extents depending on the nature of the gesture and/or speech sound. |
| first_indexed | 2024-12-22T02:43:33Z |
| format | Article |
| id | doaj.art-a389d36431a1414fa33b66b518665ab0 |
| institution | Directory Open Access Journal |
| issn | 2474-7394 |
| language | English |
| last_indexed | 2024-12-22T02:43:33Z |
| publishDate | 2017-03-01 |
| publisher | University of California Press |
| record_format | Article |
| series | Collabra: Psychology |
| spelling | doaj.art-a389d36431a1414fa33b66b518665ab02022-12-21T18:41:35ZengUniversity of California PressCollabra: Psychology2474-73942017-03-013110.1525/collabra.7642Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic ContrastsSpencer Kelly0April Bailey1Yukari Hirata2Department of Psychology, Center for Language and Brain, Colgate UniversityDepartment of Psychology, Yale UniversityDepartment of East Asian Languages and Literatures, Center for Language and Brain, Colgate UniversityIt is well established that hand gestures affect comprehension and learning of semantic aspects of a foreign language (FL). However, much less is known about the role of hand gestures in lower-level language processes, such as perception of phonemes. To address this gap, we explored the role that metaphoric gestures play in perceiving FL speech sounds that varied on two dimensions: length and intonation. English speaking adults listened to Japanese length contrasts and sentence-final intonational distinctions in the context of congruent, incongruent and no gestures. For intonational contrasts, identification was more accurate for congruent gestures and less accurate for incongruent gestures relative to the baseline no gesture condition. However, for the length contrasts, there was no such clear and consistent pattern, and in fact, congruent gestures made speech processing more effortful. We conclude that metaphoric gestures help with some—but not all—novel speech sounds in a FL, suggesting that gesture and speech are phonemically integrated to differing extents depending on the nature of the gesture and/or speech sound.https://www.collabra.org/articles/76foreign languagespeechhand gesturephonememultimodallength contrastsintonational contrastsperception |
| spellingShingle | Spencer Kelly April Bailey Yukari Hirata Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts Collabra: Psychology foreign language speech hand gesture phoneme multimodal length contrasts intonational contrasts perception |
| title | Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts |
| title_full | Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts |
| title_fullStr | Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts |
| title_full_unstemmed | Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts |
| title_short | Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts |
| title_sort | metaphoric gestures facilitate perception of intonation more than length in auditory judgments of non native phonemic contrasts |
| topic | foreign language speech hand gesture phoneme multimodal length contrasts intonational contrasts perception |
| url | https://www.collabra.org/articles/76 |
| work_keys_str_mv | AT spencerkelly metaphoricgesturesfacilitateperceptionofintonationmorethanlengthinauditoryjudgmentsofnonnativephonemiccontrasts AT aprilbailey metaphoricgesturesfacilitateperceptionofintonationmorethanlengthinauditoryjudgmentsofnonnativephonemiccontrasts AT yukarihirata metaphoricgesturesfacilitateperceptionofintonationmorethanlengthinauditoryjudgmentsofnonnativephonemiccontrasts |