IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism
In the context in which it was demonstrated that humanoid robots are efficient in helping children diagnosed with autism in exploring their affective state, this paper underlines and proves the efficiency of a previously developed machine learning-based mobile application called PandaSays, which was...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/7/2528 |
| _version_ | 1797437784858820608 |
|---|---|
| author | Aura-Loredana Popescu Nirvana Popescu Ciprian Dobre Elena-Simona Apostol Decebal Popescu |
| author_facet | Aura-Loredana Popescu Nirvana Popescu Ciprian Dobre Elena-Simona Apostol Decebal Popescu |
| author_sort | Aura-Loredana Popescu |
| collection | DOAJ |
| description | In the context in which it was demonstrated that humanoid robots are efficient in helping children diagnosed with autism in exploring their affective state, this paper underlines and proves the efficiency of a previously developed machine learning-based mobile application called PandaSays, which was improved and integrated with an Alpha 1 Pro robot, and discusses performance evaluations using deep convolutional neural networks and residual neural networks. The model trained with MobileNet convolutional neural network had an accuracy of 56.25%, performing better than ResNet50 and VGG16. A strategy for commanding the Alpha 1 Pro robot without its native application was also established and a robot module was developed that includes the communication protocols with the application PandaSays. The output of the machine learning algorithm involved in PandaSays is sent to the humanoid robot to execute some actions as singing, dancing, and so on. Alpha 1 Pro has its own programming language—Blockly—and, in order to give the robot specific commands, Bluetooth programming is used, with the help of a Raspberry Pi. Therefore, the robot motions can be controlled based on the corresponding protocols. The tests have proved the robustness of the whole solution. |
| first_indexed | 2024-03-09T11:26:38Z |
| format | Article |
| id | doaj.art-787af83d18b749f2bb066485a6b029bd |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T11:26:38Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-787af83d18b749f2bb066485a6b029bd2023-12-01T00:00:11ZengMDPI AGSensors1424-82202022-03-01227252810.3390/s22072528IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with AutismAura-Loredana Popescu0Nirvana Popescu1Ciprian Dobre2Elena-Simona Apostol3Decebal Popescu4Computer Science Department, University POLITEHNICA of Bucharest, RO-060042 Bucharest, RomaniaComputer Science Department, University POLITEHNICA of Bucharest, RO-060042 Bucharest, RomaniaComputer Science Department, University POLITEHNICA of Bucharest, RO-060042 Bucharest, RomaniaComputer Science Department, University POLITEHNICA of Bucharest, RO-060042 Bucharest, RomaniaComputer Science Department, University POLITEHNICA of Bucharest, RO-060042 Bucharest, RomaniaIn the context in which it was demonstrated that humanoid robots are efficient in helping children diagnosed with autism in exploring their affective state, this paper underlines and proves the efficiency of a previously developed machine learning-based mobile application called PandaSays, which was improved and integrated with an Alpha 1 Pro robot, and discusses performance evaluations using deep convolutional neural networks and residual neural networks. The model trained with MobileNet convolutional neural network had an accuracy of 56.25%, performing better than ResNet50 and VGG16. A strategy for commanding the Alpha 1 Pro robot without its native application was also established and a robot module was developed that includes the communication protocols with the application PandaSays. The output of the machine learning algorithm involved in PandaSays is sent to the humanoid robot to execute some actions as singing, dancing, and so on. Alpha 1 Pro has its own programming language—Blockly—and, in order to give the robot specific commands, Bluetooth programming is used, with the help of a Raspberry Pi. Therefore, the robot motions can be controlled based on the corresponding protocols. The tests have proved the robustness of the whole solution.https://www.mdpi.com/1424-8220/22/7/2528neural networksautism spectrum disorderhumanoid robotsdrawing interpretation |
| spellingShingle | Aura-Loredana Popescu Nirvana Popescu Ciprian Dobre Elena-Simona Apostol Decebal Popescu IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism Sensors neural networks autism spectrum disorder humanoid robots drawing interpretation |
| title | IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism |
| title_full | IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism |
| title_fullStr | IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism |
| title_full_unstemmed | IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism |
| title_short | IoT and AI-Based Application for Automatic Interpretation of the Affective State of Children Diagnosed with Autism |
| title_sort | iot and ai based application for automatic interpretation of the affective state of children diagnosed with autism |
| topic | neural networks autism spectrum disorder humanoid robots drawing interpretation |
| url | https://www.mdpi.com/1424-8220/22/7/2528 |
| work_keys_str_mv | AT auraloredanapopescu iotandaibasedapplicationforautomaticinterpretationoftheaffectivestateofchildrendiagnosedwithautism AT nirvanapopescu iotandaibasedapplicationforautomaticinterpretationoftheaffectivestateofchildrendiagnosedwithautism AT cipriandobre iotandaibasedapplicationforautomaticinterpretationoftheaffectivestateofchildrendiagnosedwithautism AT elenasimonaapostol iotandaibasedapplicationforautomaticinterpretationoftheaffectivestateofchildrendiagnosedwithautism AT decebalpopescu iotandaibasedapplicationforautomaticinterpretationoftheaffectivestateofchildrendiagnosedwithautism |