ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications
To train deep learning models for vision-based action recognition of elders’ daily activities, we need large-scale activity datasets acquired under various daily living environments and conditions. However, most public datasets used in human action recognition either differ from or have l...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9324837/ |
| _version_ | 1797938211363749888 |
|---|---|
| author | Hochul Hwang Cheongjae Jang Geonwoo Park Junghyun Cho Ig-Jae Kim |
| author_facet | Hochul Hwang Cheongjae Jang Geonwoo Park Junghyun Cho Ig-Jae Kim |
| author_sort | Hochul Hwang |
| collection | DOAJ |
| description | To train deep learning models for vision-based action recognition of elders’ daily activities, we need large-scale activity datasets acquired under various daily living environments and conditions. However, most public datasets used in human action recognition either differ from or have limited coverage of elders’ activities in many aspects, making it challenging to recognize elders’ daily activities well by only utilizing existing datasets. Recently, such limitations of available datasets have actively been compensated by generating synthetic data from realistic simulation environments and using those data to train deep learning models. In this paper, based on these ideas we develop ElderSim, an action simulation platform that can generate synthetic data on elders’ daily activities. For 55 kinds of frequent daily activities of the elders, ElderSim generates realistic motions of synthetic characters with various adjustable data-generating options and provides different output modalities including RGB videos, two- and three-dimensional skeleton trajectories. We then generate KIST SynADL, a large-scale synthetic dataset of elders’ activities of daily living, from ElderSim and use the data in addition to real datasets to train three state-of-the-art human action recognition models. From the experiments following several newly proposed scenarios that assume different real and synthetic dataset configurations for training, we observe a noticeable performance improvement by augmenting our synthetic data. We also offer guidance with insights for the effective utilization of synthetic data to help recognize elders’ daily activities. |
| first_indexed | 2024-04-10T18:56:13Z |
| format | Article |
| id | doaj.art-d5257e85ff2241949c7ac7f9d67e5cf3 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T18:56:13Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d5257e85ff2241949c7ac7f9d67e5cf32023-02-01T00:00:30ZengIEEEIEEE Access2169-35362023-01-01119279929410.1109/ACCESS.2021.30518429324837ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare ApplicationsHochul Hwang0https://orcid.org/0000-0002-3199-7208Cheongjae Jang1Geonwoo Park2https://orcid.org/0000-0003-3712-638XJunghyun Cho3https://orcid.org/0000-0003-1913-8037Ig-Jae Kim4https://orcid.org/0000-0002-2741-7047Artificial Intelligence and Robotics Institute, Korea Institute of Science and Technology (KIST), Seoul, South KoreaArtificial Intelligence and Robotics Institute, Korea Institute of Science and Technology (KIST), Seoul, South KoreaArtificial Intelligence and Robotics Institute, Korea Institute of Science and Technology (KIST), Seoul, South KoreaArtificial Intelligence and Robotics Institute, Korea Institute of Science and Technology (KIST), Seoul, South KoreaArtificial Intelligence and Robotics Institute, Korea Institute of Science and Technology (KIST), Seoul, South KoreaTo train deep learning models for vision-based action recognition of elders’ daily activities, we need large-scale activity datasets acquired under various daily living environments and conditions. However, most public datasets used in human action recognition either differ from or have limited coverage of elders’ activities in many aspects, making it challenging to recognize elders’ daily activities well by only utilizing existing datasets. Recently, such limitations of available datasets have actively been compensated by generating synthetic data from realistic simulation environments and using those data to train deep learning models. In this paper, based on these ideas we develop ElderSim, an action simulation platform that can generate synthetic data on elders’ daily activities. For 55 kinds of frequent daily activities of the elders, ElderSim generates realistic motions of synthetic characters with various adjustable data-generating options and provides different output modalities including RGB videos, two- and three-dimensional skeleton trajectories. We then generate KIST SynADL, a large-scale synthetic dataset of elders’ activities of daily living, from ElderSim and use the data in addition to real datasets to train three state-of-the-art human action recognition models. From the experiments following several newly proposed scenarios that assume different real and synthetic dataset configurations for training, we observe a noticeable performance improvement by augmenting our synthetic data. We also offer guidance with insights for the effective utilization of synthetic data to help recognize elders’ daily activities.https://ieeexplore.ieee.org/document/9324837/Classification algorithmscomputer graphicscomputer simulationcomputer visionsupervised learning |
| spellingShingle | Hochul Hwang Cheongjae Jang Geonwoo Park Junghyun Cho Ig-Jae Kim ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications IEEE Access Classification algorithms computer graphics computer simulation computer vision supervised learning |
| title | ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications |
| title_full | ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications |
| title_fullStr | ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications |
| title_full_unstemmed | ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications |
| title_short | ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications |
| title_sort | eldersim a synthetic data generation platform for human action recognition in eldercare applications |
| topic | Classification algorithms computer graphics computer simulation computer vision supervised learning |
| url | https://ieeexplore.ieee.org/document/9324837/ |
| work_keys_str_mv | AT hochulhwang eldersimasyntheticdatagenerationplatformforhumanactionrecognitionineldercareapplications AT cheongjaejang eldersimasyntheticdatagenerationplatformforhumanactionrecognitionineldercareapplications AT geonwoopark eldersimasyntheticdatagenerationplatformforhumanactionrecognitionineldercareapplications AT junghyuncho eldersimasyntheticdatagenerationplatformforhumanactionrecognitionineldercareapplications AT igjaekim eldersimasyntheticdatagenerationplatformforhumanactionrecognitionineldercareapplications |