Tactual Display of Consonant Voicing to Supplement Lipreading
Thesis Supervisor: Nathaniel I. Durlach, Senior Research Scientist. Thesis Supervisor: Charlotte M. Reed, Senior Research Scientist.
| Main Author: | |
|---|---|
| Format: | Technical Report |
| Language: | English |
| Published: |
2004
|
| Online Access: | http://hdl.handle.net/1721.1/6568 |
| _version_ | 1811074342244581376 |
|---|---|
| author | Yuan, Hanfeng |
| author_facet | Yuan, Hanfeng |
| author_sort | Yuan, Hanfeng |
| collection | MIT |
| description | Thesis Supervisor: Nathaniel I. Durlach, Senior Research Scientist. Thesis Supervisor: Charlotte M. Reed, Senior Research Scientist. |
| first_indexed | 2024-09-23T09:47:33Z |
| format | Technical Report |
| id | mit-1721.1/6568 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T09:47:33Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/65682019-04-12T08:39:16Z Tactual Display of Consonant Voicing to Supplement Lipreading Yuan, Hanfeng Thesis Supervisor: Nathaniel I. Durlach, Senior Research Scientist. Thesis Supervisor: Charlotte M. Reed, Senior Research Scientist. This research is concerned with the development of tactual displays to supplement the information available through lipreading. Because voicing carries a high informational load in speech and is not well transmitted through lipreading, the efforts are focused on providing tactual displays of voicing to supplement the information available on the lips of the talker. This research includes exploration of 1) signal-processing schemes to extract information about voicing from the acoustic speech signal, 2) methods of displaying this information through a multi-finger tactual display, and 3) perceptual evaluations of voicing reception through the tactual display alone (T), lipreading alone (L), and the combined condition (L+T). Signal processing for the extraction of voicing information used amplitude-envelope signals derived from filtered bands of speech (i.e., envelopes derived from a lowpass-filtered band at 350 Hz and from a highpass-filtered band at 3000 Hz). Acoustic measurements made on the envelope signals of a set of 16 initial consonants represented through multiple tokens of C1VC2 syllables indicate that the onset-timing difference between the low- and high-frequency envelopes (EOA: envelope-onset asynchrony) provides a reliable and robust cue for distinguishing voiced from voiceless consonants. This acoustic cue was presented through a two-finger tactual display such that the envelope of the high-frequency band was used to modulate a 250-Hz carrier signal delivered to the index finger (250-I) and the envelope of the low-frequency band was used to modulate a 50-Hz carrier delivered to the thumb (50T). The temporal-onset order threshold for these two signals, measured with roving signal amplitude and duration, averaged 34 msec, sufficiently small for use of the EOA cue. Perceptual evaluations of the tactual display of EOA with speech signal indicated: 1) that the cue was highly effective for discrimination of pairs of voicing contrasts; 2) that the identification of 16 consonants was improved by roughly 15 percentage points with the addition of the tactual cue over L alone; and 3) that no improvements in L+T over L were observed for reception of words in sentences, indicating the need for further training on this task 2004-10-07T15:17:21Z 2004-10-07T15:17:21Z 2004-10-07T15:17:21Z Technical Report http://hdl.handle.net/1721.1/6568 en Technical Report (Massachusetts Institute of Technology, Research Laboratory of Electronics);669 2702462 bytes application/pdf application/pdf |
| spellingShingle | Yuan, Hanfeng Tactual Display of Consonant Voicing to Supplement Lipreading |
| title | Tactual Display of Consonant Voicing to Supplement Lipreading |
| title_full | Tactual Display of Consonant Voicing to Supplement Lipreading |
| title_fullStr | Tactual Display of Consonant Voicing to Supplement Lipreading |
| title_full_unstemmed | Tactual Display of Consonant Voicing to Supplement Lipreading |
| title_short | Tactual Display of Consonant Voicing to Supplement Lipreading |
| title_sort | tactual display of consonant voicing to supplement lipreading |
| url | http://hdl.handle.net/1721.1/6568 |
| work_keys_str_mv | AT yuanhanfeng tactualdisplayofconsonantvoicingtosupplementlipreading |