Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception
We investigated how the physical differences associated with the articulation of speech affect the temporal aspects of audiovisual speech perception. Video clips of consonants and vowels uttered by three different speakers were presented. The video clips were analyzed using an auditory-visual signal...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Frontiers Media
2012
|
| _version_ | 1826279263803801600 |
|---|---|
| author | Vatakis, A Maragos, P Rodomagoulakis, I Spence, C |
| author_facet | Vatakis, A Maragos, P Rodomagoulakis, I Spence, C |
| author_sort | Vatakis, A |
| collection | OXFORD |
| description | We investigated how the physical differences associated with the articulation of speech affect the temporal aspects of audiovisual speech perception. Video clips of consonants and vowels uttered by three different speakers were presented. The video clips were analyzed using an auditory-visual signal saliency model in order to compare signal saliency and behavioral data. Participants made temporal order judgments (TOJs) regarding which speech-stream (auditory or visual) had been presented first. The sensitivity of participants' TOJs and the point of subjective simultaneity (PSS) were analyzed as a function of the place, manner of articulation, and voicing for consonants, and the height/backness of the tongue and lip-roundedness for vowels. We expected that in the case of the place of articulation and roundedness, where the visual-speech signal is more salient, temporal perception of speech would be modulated by the visual-speech signal. No such effect was expected for the manner of articulation or height. The results demonstrate that for place and manner of articulation, participants' temporal percept was affected (although not always significantly) by highly-salient speech-signals with the visual-signals requiring smaller visual-leads at the PSS. This was not the case when height was evaluated. These findings suggest that in the case of audiovisual speech perception, a highly salient visual-speech signal may lead to higher probabilities regarding the identity of the auditory-signal that modulate the temporal window of multisensory integration of the speech-stimulus. |
| first_indexed | 2024-03-06T23:56:08Z |
| format | Journal article |
| id | oxford-uuid:7448fad1-8011-4acd-bc90-d423b627916b |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T23:56:08Z |
| publishDate | 2012 |
| publisher | Frontiers Media |
| record_format | dspace |
| spelling | oxford-uuid:7448fad1-8011-4acd-bc90-d423b627916b2022-03-26T20:01:45ZAssessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perceptionJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:7448fad1-8011-4acd-bc90-d423b627916bEnglishSymplectic Elements at OxfordFrontiers Media2012Vatakis, AMaragos, PRodomagoulakis, ISpence, CWe investigated how the physical differences associated with the articulation of speech affect the temporal aspects of audiovisual speech perception. Video clips of consonants and vowels uttered by three different speakers were presented. The video clips were analyzed using an auditory-visual signal saliency model in order to compare signal saliency and behavioral data. Participants made temporal order judgments (TOJs) regarding which speech-stream (auditory or visual) had been presented first. The sensitivity of participants' TOJs and the point of subjective simultaneity (PSS) were analyzed as a function of the place, manner of articulation, and voicing for consonants, and the height/backness of the tongue and lip-roundedness for vowels. We expected that in the case of the place of articulation and roundedness, where the visual-speech signal is more salient, temporal perception of speech would be modulated by the visual-speech signal. No such effect was expected for the manner of articulation or height. The results demonstrate that for place and manner of articulation, participants' temporal percept was affected (although not always significantly) by highly-salient speech-signals with the visual-signals requiring smaller visual-leads at the PSS. This was not the case when height was evaluated. These findings suggest that in the case of audiovisual speech perception, a highly salient visual-speech signal may lead to higher probabilities regarding the identity of the auditory-signal that modulate the temporal window of multisensory integration of the speech-stimulus. |
| spellingShingle | Vatakis, A Maragos, P Rodomagoulakis, I Spence, C Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| title | Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| title_full | Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| title_fullStr | Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| title_full_unstemmed | Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| title_short | Assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| title_sort | assessing the effect of physical differences in the articulation of consonants and vowels on audiovisual temporal perception |
| work_keys_str_mv | AT vatakisa assessingtheeffectofphysicaldifferencesinthearticulationofconsonantsandvowelsonaudiovisualtemporalperception AT maragosp assessingtheeffectofphysicaldifferencesinthearticulationofconsonantsandvowelsonaudiovisualtemporalperception AT rodomagoulakisi assessingtheeffectofphysicaldifferencesinthearticulationofconsonantsandvowelsonaudiovisualtemporalperception AT spencec assessingtheeffectofphysicaldifferencesinthearticulationofconsonantsandvowelsonaudiovisualtemporalperception |