Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation
SA Conference Papers ’24, December 03–06, 2024, Tokyo, Japan
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
ACM|SIGGRAPH Asia 2024 Conference Papers
2025
|
| Online Access: | https://hdl.handle.net/1721.1/158128 |
| _version_ | 1824458030962442240 |
|---|---|
| author | Caren, Matthew Chandra, Kartik Tenenbaum, Joshua Ragan-Kelley, Jonathan Ma, Karima |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Caren, Matthew Chandra, Kartik Tenenbaum, Joshua Ragan-Kelley, Jonathan Ma, Karima |
| author_sort | Caren, Matthew |
| collection | MIT |
| description | SA Conference Papers ’24, December 03–06, 2024, Tokyo, Japan |
| first_indexed | 2025-02-19T04:19:25Z |
| format | Article |
| id | mit-1721.1/158128 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2025-02-19T04:19:25Z |
| publishDate | 2025 |
| publisher | ACM|SIGGRAPH Asia 2024 Conference Papers |
| record_format | dspace |
| spelling | mit-1721.1/1581282025-01-29T19:32:55Z Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation Caren, Matthew Chandra, Kartik Tenenbaum, Joshua Ragan-Kelley, Jonathan Ma, Karima Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences SA Conference Papers ’24, December 03–06, 2024, Tokyo, Japan We present a method for automatically producing human-like vocal imitations of sounds: the equivalent of “sketching,” but for auditory rather than visual representation. Starting with a simulated model of the human vocal tract, we first try generating vocal imitations by tuning the model’s control parameters to make the synthesized vocalization match the target sound in terms of perceptually-salient auditory features. Then, to better match human intuitions, we apply a cognitive theory of communication to take into account how human speakers reason strategically about their listeners. Finally, we show through several experiments and user studies that when we add this type of communicative reasoning to our method, it aligns with human intuitions better than matching auditory features alone does. This observation has broad implications for the study of depiction in computer graphics. 2025-01-29T19:32:54Z 2025-01-29T19:32:54Z 2024-12-03 2025-01-01T08:51:10Z Article http://purl.org/eprint/type/ConferencePaper 979-8-4007-1131-2 https://hdl.handle.net/1721.1/158128 Caren, Matthew, Chandra, Kartik, Tenenbaum, Joshua, Ragan-Kelley, Jonathan and Ma, Karima. 2024. "Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation." PUBLISHER_CC en https://doi.org/10.1145/3680528.3687679 Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ The author(s) application/pdf ACM|SIGGRAPH Asia 2024 Conference Papers Association for Computing Machinery |
| spellingShingle | Caren, Matthew Chandra, Kartik Tenenbaum, Joshua Ragan-Kelley, Jonathan Ma, Karima Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation |
| title | Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation |
| title_full | Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation |
| title_fullStr | Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation |
| title_full_unstemmed | Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation |
| title_short | Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation |
| title_sort | sketching with your voice non phonorealistic rendering of sounds via vocal imitation |
| url | https://hdl.handle.net/1721.1/158128 |
| work_keys_str_mv | AT carenmatthew sketchingwithyourvoicenonphonorealisticrenderingofsoundsviavocalimitation AT chandrakartik sketchingwithyourvoicenonphonorealisticrenderingofsoundsviavocalimitation AT tenenbaumjoshua sketchingwithyourvoicenonphonorealisticrenderingofsoundsviavocalimitation AT ragankelleyjonathan sketchingwithyourvoicenonphonorealisticrenderingofsoundsviavocalimitation AT makarima sketchingwithyourvoicenonphonorealisticrenderingofsoundsviavocalimitation |