Discourse Topic and Gestural Form
Coverbal gesture provides a channel for the visual expression of ideas. While some gestural emblems have culturally predefined forms (e.g., "thumbs up"), the relationship between gesture and meaning is, in general, not conventionalized. It is natural to ask whether such gestures can be int...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2014
|
| Online Access: | http://hdl.handle.net/1721.1/87042 https://orcid.org/0000-0002-2921-8201 https://orcid.org/0000-0001-5232-7281 |
| _version_ | 1826215824095969280 |
|---|---|
| author | Eisenstein, Jacob Barzilay, Regina Davis, Randall |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Eisenstein, Jacob Barzilay, Regina Davis, Randall |
| author_sort | Eisenstein, Jacob |
| collection | MIT |
| description | Coverbal gesture provides a channel for the visual expression of ideas. While some gestural emblems have culturally predefined forms (e.g., "thumbs up"), the relationship between gesture and meaning is, in general, not conventionalized. It is natural to ask whether such gestures can be interpreted in a speaker-independent way, or whether gestural form is determined by the speaker's idiosyncratic view of the discourse topic. We address this question using an audiovisual dataset across multiple speakers and topics. Our analysis employs a hierarchical Bayesian author-topic model, in which gestural patterns are stochastically generated by a mixture of speaker-specific and topic-specific priors. These gestural patterns are characterized using automatically-extracted visual features, based on spatio-temporal interest points. This framework detects significant cross-speaker patterns in gesture that are governed by the discourse topic, suggesting that even unstructured gesticulation can be interpreted across speakers. In addition, the success of this approach shows that the semantic characteristics of gesture can be detected via a low-level, interest point representation. |
| first_indexed | 2024-09-23T16:37:39Z |
| format | Article |
| id | mit-1721.1/87042 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T16:37:39Z |
| publishDate | 2014 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/870422022-10-02T08:33:16Z Discourse Topic and Gestural Form Eisenstein, Jacob Barzilay, Regina Davis, Randall Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Eisenstein, Jacob Barzilay, Regina Davis, Randall Coverbal gesture provides a channel for the visual expression of ideas. While some gestural emblems have culturally predefined forms (e.g., "thumbs up"), the relationship between gesture and meaning is, in general, not conventionalized. It is natural to ask whether such gestures can be interpreted in a speaker-independent way, or whether gestural form is determined by the speaker's idiosyncratic view of the discourse topic. We address this question using an audiovisual dataset across multiple speakers and topics. Our analysis employs a hierarchical Bayesian author-topic model, in which gestural patterns are stochastically generated by a mixture of speaker-specific and topic-specific priors. These gestural patterns are characterized using automatically-extracted visual features, based on spatio-temporal interest points. This framework detects significant cross-speaker patterns in gesture that are governed by the discourse topic, suggesting that even unstructured gesticulation can be interpreted across speakers. In addition, the success of this approach shows that the semantic characteristics of gesture can be detected via a low-level, interest point representation. 2014-05-16T18:49:51Z 2014-05-16T18:49:51Z 2008 Article http://purl.org/eprint/type/ConferencePaper 978-1-57735-368-3 http://hdl.handle.net/1721.1/87042 Jacob Eisenstein, Regina Barzilay, and Randall Davis. 2008. Discourse topic and gestural form. In Proceedings of the 23rd national conference on Artificial intelligence - Volume 2 (AAAI'08), Anthony Cohn (Ed.), Vol. 2. AAAI Press 836-841. © 2008, Association for the Advancement of Artificial Intelligence https://orcid.org/0000-0002-2921-8201 https://orcid.org/0000-0001-5232-7281 en_US http://dl.acm.org/citation.cfm?id=1620163.1620202 Proceedings of the 23rd national conference on Artificial intelligence - Volume 2 (AAAI '08) Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Association for Computing Machinery (ACM) ACM |
| spellingShingle | Eisenstein, Jacob Barzilay, Regina Davis, Randall Discourse Topic and Gestural Form |
| title | Discourse Topic and Gestural Form |
| title_full | Discourse Topic and Gestural Form |
| title_fullStr | Discourse Topic and Gestural Form |
| title_full_unstemmed | Discourse Topic and Gestural Form |
| title_short | Discourse Topic and Gestural Form |
| title_sort | discourse topic and gestural form |
| url | http://hdl.handle.net/1721.1/87042 https://orcid.org/0000-0002-2921-8201 https://orcid.org/0000-0001-5232-7281 |
| work_keys_str_mv | AT eisensteinjacob discoursetopicandgesturalform AT barzilayregina discoursetopicandgesturalform AT davisrandall discoursetopicandgesturalform |