Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration
The study explores the distribution and structure of multimodal clusters presenting a series of communicative moves in expositive dialogues: Request, Elaboration, and Response. We hypothesize that multimodal clustering of moves will be predetermined by the use of either common (for both participants...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Peoples’ Friendship University of Russia (RUDN University)
2023-12-01
|
| Series: | RUDN Journal of Language Studies, Semiotics and Semantics |
| Subjects: | |
| Online Access: | https://journals.rudn.ru/semiotics-semantics/article/viewFile/37650/23098 |
| _version_ | 1797338783140544512 |
|---|---|
| author | Maria I. Kiose Anna V. Leonteva Olga V. Agafonova Andrey A. Petrov |
| author_facet | Maria I. Kiose Anna V. Leonteva Olga V. Agafonova Andrey A. Petrov |
| author_sort | Maria I. Kiose |
| collection | DOAJ |
| description | The study explores the distribution and structure of multimodal clusters presenting a series of communicative moves in expositive dialogues: Request, Elaboration, and Response. We hypothesize that multimodal clustering of moves will be predetermined by the use of either common (for both participants) or novel topic elaboration as a nucleus move within the cluster. To proceed, we conduct a multimodal experiment which recorded the participants’ gesture with motion capture system (Perception Neuron Motion Capture) and gaze with eye-tracking glasses (Tobii Pro Glasses 2), as well as their speech and overall multimodal behavior with a stationary camera. The study reveals significant differences in the use of both face-oriented gaze and contactestablishing gesture as modulated by Request and Response moves within common or novel topic elaboration clusters; however, face-oriented gaze use manifests both higher frequency and diversity. Mutual face-oriented gaze prevails at the Request move preceding common topic elaboration, whereas elaborating a novel topic is found to produce a more involved gaze reaction of the listener during the Response moves. Additionally, simultaneous (by both participants) verbal move is more typical of common topic elaboration. The results evidence that social interaction and communication in expositive dialogue is processed multimodally and predetermines the role of gaze, gesture and verbal moves in communicative moves clusters. |
| first_indexed | 2024-03-08T09:36:16Z |
| format | Article |
| id | doaj.art-e2bf7fd1588c4818b63ca4bb4eb2733c |
| institution | Directory Open Access Journal |
| issn | 2313-2299 2411-1236 |
| language | English |
| last_indexed | 2024-03-08T09:36:16Z |
| publishDate | 2023-12-01 |
| publisher | Peoples’ Friendship University of Russia (RUDN University) |
| record_format | Article |
| series | RUDN Journal of Language Studies, Semiotics and Semantics |
| spelling | doaj.art-e2bf7fd1588c4818b63ca4bb4eb2733c2024-01-30T08:37:38ZengPeoples’ Friendship University of Russia (RUDN University)RUDN Journal of Language Studies, Semiotics and Semantics2313-22992411-12362023-12-011441013103510.22363/2313-2299-2023-14-4-1013-103521002Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic ElaborationMaria I. Kiose0https://orcid.org/0000-0001-7215-0604Anna V. Leonteva1Olga V. Agafonova2https://orcid.org/0000-0002-2184-163XAndrey A. Petrov3https://orcid.org/0000-0003-0368-8800Moscow State Linguistic UniversityMoscow State Linguistic UniversityMoscow State Linguistic UniversityMoscow State Linguistic UniversityThe study explores the distribution and structure of multimodal clusters presenting a series of communicative moves in expositive dialogues: Request, Elaboration, and Response. We hypothesize that multimodal clustering of moves will be predetermined by the use of either common (for both participants) or novel topic elaboration as a nucleus move within the cluster. To proceed, we conduct a multimodal experiment which recorded the participants’ gesture with motion capture system (Perception Neuron Motion Capture) and gaze with eye-tracking glasses (Tobii Pro Glasses 2), as well as their speech and overall multimodal behavior with a stationary camera. The study reveals significant differences in the use of both face-oriented gaze and contactestablishing gesture as modulated by Request and Response moves within common or novel topic elaboration clusters; however, face-oriented gaze use manifests both higher frequency and diversity. Mutual face-oriented gaze prevails at the Request move preceding common topic elaboration, whereas elaborating a novel topic is found to produce a more involved gaze reaction of the listener during the Response moves. Additionally, simultaneous (by both participants) verbal move is more typical of common topic elaboration. The results evidence that social interaction and communication in expositive dialogue is processed multimodally and predetermines the role of gaze, gesture and verbal moves in communicative moves clusters.https://journals.rudn.ru/semiotics-semantics/article/viewFile/37650/23098expositive discoursedialoguemultimodalitycommunicative moverequestcommon topic elaborationnovel topic elaborationresponsegazespeechgesture |
| spellingShingle | Maria I. Kiose Anna V. Leonteva Olga V. Agafonova Andrey A. Petrov Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration RUDN Journal of Language Studies, Semiotics and Semantics expositive discourse dialogue multimodality communicative move request common topic elaboration novel topic elaboration response gaze speech gesture |
| title | Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration |
| title_full | Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration |
| title_fullStr | Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration |
| title_full_unstemmed | Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration |
| title_short | Multimodal Communicative Moves in Expositive Dialogue: Common and Novel Topic Elaboration |
| title_sort | multimodal communicative moves in expositive dialogue common and novel topic elaboration |
| topic | expositive discourse dialogue multimodality communicative move request common topic elaboration novel topic elaboration response gaze speech gesture |
| url | https://journals.rudn.ru/semiotics-semantics/article/viewFile/37650/23098 |
| work_keys_str_mv | AT mariaikiose multimodalcommunicativemovesinexpositivedialoguecommonandnoveltopicelaboration AT annavleonteva multimodalcommunicativemovesinexpositivedialoguecommonandnoveltopicelaboration AT olgavagafonova multimodalcommunicativemovesinexpositivedialoguecommonandnoveltopicelaboration AT andreyapetrov multimodalcommunicativemovesinexpositivedialoguecommonandnoveltopicelaboration |