Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting
BackgroundRobots offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Determining the optimal motion of robots when interacting with individuals with ASD is important for achieving more natural human-robot interactions and for exploiting the full potential of...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-02-01
|
| Series: | Frontiers in Psychiatry |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyt.2024.1249000/full |
| _version_ | 1797323574482042880 |
|---|---|
| author | Shunta Konishi Masaki Kuwata Yoshio Matsumoto Yuichiro Yoshikawa Keiji Takata Hideyuki Haraguchi Azusa Kudo Hiroshi Ishiguro Hirokazu Kumazaki Hirokazu Kumazaki |
| author_facet | Shunta Konishi Masaki Kuwata Yoshio Matsumoto Yuichiro Yoshikawa Keiji Takata Hideyuki Haraguchi Azusa Kudo Hiroshi Ishiguro Hirokazu Kumazaki Hirokazu Kumazaki |
| author_sort | Shunta Konishi |
| collection | DOAJ |
| description | BackgroundRobots offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Determining the optimal motion of robots when interacting with individuals with ASD is important for achieving more natural human-robot interactions and for exploiting the full potential of robotic interventions. Most prior studies have used supervised machine learning (ML) of user behavioral data to enable robot perception of affective states (i.e., arousal and valence) and engagement. It has previously been suggested that including personal demographic information in the identification of individuals with ASD is important for developing an automated system to perceive individual affective states and engagement. In this study, we hypothesized that assessing self-administered questionnaire data would contribute to the development of an automated estimation of the affective state and engagement when individuals with ASD are interviewed by an Android robot, which will be linked to implementing long-term interventions and maintaining the motivation of participants.MethodsParticipants sat across a table from an android robot that played the role of the interviewer. Each participant underwent a mock job interview. Twenty-five participants with ASD (males 22, females 3, average chronological age = 22.8, average IQ = 94.04) completed the experiment. We collected multimodal data (i.e., audio, motion, gaze, and self-administered questionnaire data) to train a model to correctly classify the state of individuals with ASD when interviewed by an android robot. We demonstrated the technical feasibility of using ML to enable robot perception of affect and engagement of individuals with ASD based on multimodal data.ResultsFor arousal and engagement, the area under the curve (AUC) values of the model estimates and expert coding were relatively high. Overall, the AUC values of arousal, valence, and engagement were improved by including self-administered questionnaire data in the classification.DiscussionThese findings support the hypothesis that assessing self-administered questionnaire data contributes to the development of an automated estimation of an individual’s affective state and engagement. Given the efficacy of including self-administered questionnaire data, future studies should confirm the effectiveness of such long-term intervention with a robot to maintain participants’ motivation based on the proposed method of emotion estimation. |
| first_indexed | 2024-03-08T05:30:34Z |
| format | Article |
| id | doaj.art-0f486e8b06e84f9f9314d9921d329408 |
| institution | Directory Open Access Journal |
| issn | 1664-0640 |
| language | English |
| last_indexed | 2024-03-08T05:30:34Z |
| publishDate | 2024-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychiatry |
| spelling | doaj.art-0f486e8b06e84f9f9314d9921d3294082024-02-06T04:57:31ZengFrontiers Media S.A.Frontiers in Psychiatry1664-06402024-02-011510.3389/fpsyt.2024.12490001249000Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview settingShunta Konishi0Masaki Kuwata1Yoshio Matsumoto2Yuichiro Yoshikawa3Keiji Takata4Hideyuki Haraguchi5Azusa Kudo6Hiroshi Ishiguro7Hirokazu Kumazaki8Hirokazu Kumazaki9Department of Human and Engineered Environmental Studies, Graduate School of Frontier Sciences, The University of Tokyo, Chiba, JapanDepartment of Human and Engineered Environmental Studies, Graduate School of Frontier Sciences, The University of Tokyo, Chiba, JapanDepartment of Medical and Robotic Engineering Design, Faculty of Advanced Engineering, Tokyo University of Science, Tokyo, JapanDepartment of Systems Innovation, Graduate School of Engineering Science, Osaka University, Osaka, JapanNational Center of Neurology and Psychiatry, Department of Preventive Intervention for Psychiatric Disorders, National Institute of Mental Health, Tokyo, JapanNational Center of Neurology and Psychiatry, Department of Preventive Intervention for Psychiatric Disorders, National Institute of Mental Health, Tokyo, JapanDepartment of Neuropsychiatry, Graduate School of Biomedical Sciences, Nagasaki University, Nagasaki, JapanDepartment of Systems Innovation, Graduate School of Engineering Science, Osaka University, Osaka, JapanDepartment of Neuropsychiatry, Graduate School of Biomedical Sciences, Nagasaki University, Nagasaki, JapanCollege of Science and Engineering, Kanazawa University, Kanazawa, JapanBackgroundRobots offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Determining the optimal motion of robots when interacting with individuals with ASD is important for achieving more natural human-robot interactions and for exploiting the full potential of robotic interventions. Most prior studies have used supervised machine learning (ML) of user behavioral data to enable robot perception of affective states (i.e., arousal and valence) and engagement. It has previously been suggested that including personal demographic information in the identification of individuals with ASD is important for developing an automated system to perceive individual affective states and engagement. In this study, we hypothesized that assessing self-administered questionnaire data would contribute to the development of an automated estimation of the affective state and engagement when individuals with ASD are interviewed by an Android robot, which will be linked to implementing long-term interventions and maintaining the motivation of participants.MethodsParticipants sat across a table from an android robot that played the role of the interviewer. Each participant underwent a mock job interview. Twenty-five participants with ASD (males 22, females 3, average chronological age = 22.8, average IQ = 94.04) completed the experiment. We collected multimodal data (i.e., audio, motion, gaze, and self-administered questionnaire data) to train a model to correctly classify the state of individuals with ASD when interviewed by an android robot. We demonstrated the technical feasibility of using ML to enable robot perception of affect and engagement of individuals with ASD based on multimodal data.ResultsFor arousal and engagement, the area under the curve (AUC) values of the model estimates and expert coding were relatively high. Overall, the AUC values of arousal, valence, and engagement were improved by including self-administered questionnaire data in the classification.DiscussionThese findings support the hypothesis that assessing self-administered questionnaire data contributes to the development of an automated estimation of an individual’s affective state and engagement. Given the efficacy of including self-administered questionnaire data, future studies should confirm the effectiveness of such long-term intervention with a robot to maintain participants’ motivation based on the proposed method of emotion estimation.https://www.frontiersin.org/articles/10.3389/fpsyt.2024.1249000/fullautism spectrum disordersmachine learningself-administered questionnaireaffective stateautomated estimation |
| spellingShingle | Shunta Konishi Masaki Kuwata Yoshio Matsumoto Yuichiro Yoshikawa Keiji Takata Hideyuki Haraguchi Azusa Kudo Hiroshi Ishiguro Hirokazu Kumazaki Hirokazu Kumazaki Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting Frontiers in Psychiatry autism spectrum disorders machine learning self-administered questionnaire affective state automated estimation |
| title | Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting |
| title_full | Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting |
| title_fullStr | Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting |
| title_full_unstemmed | Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting |
| title_short | Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting |
| title_sort | self administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting |
| topic | autism spectrum disorders machine learning self-administered questionnaire affective state automated estimation |
| url | https://www.frontiersin.org/articles/10.3389/fpsyt.2024.1249000/full |
| work_keys_str_mv | AT shuntakonishi selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT masakikuwata selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT yoshiomatsumoto selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT yuichiroyoshikawa selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT keijitakata selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT hideyukiharaguchi selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT azusakudo selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT hiroshiishiguro selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT hirokazukumazaki selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting AT hirokazukumazaki selfadministeredquestionnairesenhanceemotionestimationofindividualswithautismspectrumdisordersinaroboticinterviewsetting |