Real-time comprehensive sociometrics for two-person dialogs
A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by soc...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2013
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/101166 http://hdl.handle.net/10220/18315 |
| _version_ | 1811686437640208384 |
|---|---|
| author | Dauwels, Shoko Rasheed, Umer Tahir, Yasir Dauwels, Justin Thalmann, Daniel |
| author2 | Nanyang Business School |
| author_facet | Nanyang Business School Dauwels, Shoko Rasheed, Umer Tahir, Yasir Dauwels, Justin Thalmann, Daniel |
| author_sort | Dauwels, Shoko |
| collection | NTU |
| description | A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by sociometrics including level of interest, agreement, and dominance. Such quantitative measures can be used to provide real-time feedback to the speakers, for instance, to alarm to speaker when the voice is too strong (speaking mannerism), or when the conversation is not proceeding well due to disagreements or numerous interruptions (social behavior). In the proposed approach, machine learning algorithms are designed to compute the sociometrics (level of interest, agreement, and dominance) in real-time from combinations of low-level speech metrics. To this end, a corpus of 150 brief two-person dialogs in English was collected. Several experts assessed the sociometrics for each of those dialogs. Next, the resulting annotated dialogs are used to train the machine learning algorithms in a supervised manner. Through this training procedure, the algorithms learn how the sociometrics depend on the low-level speech metrics, and consequently, are able to compute the sociometrics from speech recordings in an automated fashion, without further help of experts. Numerical tests through leave-one-out cross-validation indicate that the accuracy of the algorithms for inferring the sociometrics is in the range of 80-90%. In future, those reliable predictions can be the key to real-time sociofeedback, where speakers will be provided feedback in real-time about their behavior in an ongoing discussion. Such technology may be helpful in many contexts, for instance in group meetings, counseling, or executive training. |
| first_indexed | 2024-10-01T05:00:24Z |
| format | Conference Paper |
| id | ntu-10356/101166 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:00:24Z |
| publishDate | 2013 |
| record_format | dspace |
| spelling | ntu-10356/1011662023-05-19T06:44:44Z Real-time comprehensive sociometrics for two-person dialogs Dauwels, Shoko Rasheed, Umer Tahir, Yasir Dauwels, Justin Thalmann, Daniel Nanyang Business School School of Electrical and Electronic Engineering International Workshop, Human Behavior Understanding (4th : 2013 : Barcelona, Spain) Institute for Media Innovation DRNTU::Social sciences::Sociology::Social behavior A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by sociometrics including level of interest, agreement, and dominance. Such quantitative measures can be used to provide real-time feedback to the speakers, for instance, to alarm to speaker when the voice is too strong (speaking mannerism), or when the conversation is not proceeding well due to disagreements or numerous interruptions (social behavior). In the proposed approach, machine learning algorithms are designed to compute the sociometrics (level of interest, agreement, and dominance) in real-time from combinations of low-level speech metrics. To this end, a corpus of 150 brief two-person dialogs in English was collected. Several experts assessed the sociometrics for each of those dialogs. Next, the resulting annotated dialogs are used to train the machine learning algorithms in a supervised manner. Through this training procedure, the algorithms learn how the sociometrics depend on the low-level speech metrics, and consequently, are able to compute the sociometrics from speech recordings in an automated fashion, without further help of experts. Numerical tests through leave-one-out cross-validation indicate that the accuracy of the algorithms for inferring the sociometrics is in the range of 80-90%. In future, those reliable predictions can be the key to real-time sociofeedback, where speakers will be provided feedback in real-time about their behavior in an ongoing discussion. Such technology may be helpful in many contexts, for instance in group meetings, counseling, or executive training. Accepted version 2013-12-18T06:32:22Z 2019-12-06T20:34:29Z 2013-12-18T06:32:22Z 2019-12-06T20:34:29Z 2013 2013 Conference Paper Magnenat-Thalmann, N., Rasheed, U., Tahir, Y., Dauwels, S., Dauwels, J., & Thalmann, D. (2013). Real-time comprehensive sociometrics for two-person dialogs. Proceedings of the 4th International Workshop, Human Behavior Understanding (HBU 2013), LNCS 8212, 196-208. https://hdl.handle.net/10356/101166 http://hdl.handle.net/10220/18315 10.1007/978-3-319-02714-2_17 en © 2013 Springer International Publishing Switzerland. This is the author created version of a work that has been peer reviewed and accepted for publication by Proceedings of the 4th International Workshop, Human Behavior Understanding (HBU 2013), Springer International Publishing Switzerland. It incorporates referee’s comments but changes resulting from the publishing process, such as copyediting, structural formatting, may not be reflected in this document. The published version is available at: [http://dx.doi.org/10.1007/978-3-319-02714-2_17]. 13 p. application/pdf |
| spellingShingle | DRNTU::Social sciences::Sociology::Social behavior Dauwels, Shoko Rasheed, Umer Tahir, Yasir Dauwels, Justin Thalmann, Daniel Real-time comprehensive sociometrics for two-person dialogs |
| title | Real-time comprehensive sociometrics for two-person dialogs |
| title_full | Real-time comprehensive sociometrics for two-person dialogs |
| title_fullStr | Real-time comprehensive sociometrics for two-person dialogs |
| title_full_unstemmed | Real-time comprehensive sociometrics for two-person dialogs |
| title_short | Real-time comprehensive sociometrics for two-person dialogs |
| title_sort | real time comprehensive sociometrics for two person dialogs |
| topic | DRNTU::Social sciences::Sociology::Social behavior |
| url | https://hdl.handle.net/10356/101166 http://hdl.handle.net/10220/18315 |
| work_keys_str_mv | AT dauwelsshoko realtimecomprehensivesociometricsfortwopersondialogs AT rasheedumer realtimecomprehensivesociometricsfortwopersondialogs AT tahiryasir realtimecomprehensivesociometricsfortwopersondialogs AT dauwelsjustin realtimecomprehensivesociometricsfortwopersondialogs AT thalmanndaniel realtimecomprehensivesociometricsfortwopersondialogs |