Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training
Using standard digital cameras in combination with deep learning (DL) for pose estimation is promising for the in-home and independent use of exercise games (exergames). We need to investigate to what extent such DL-based systems can provide satisfying accuracy on exergame relevant measures. Our stu...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/23/6940 |
| _version_ | 1797545782190014464 |
|---|---|
| author | Elise Klæbo Vonstad Xiaomeng Su Beatrix Vereijken Kerstin Bach Jan Harald Nilsen |
| author_facet | Elise Klæbo Vonstad Xiaomeng Su Beatrix Vereijken Kerstin Bach Jan Harald Nilsen |
| author_sort | Elise Klæbo Vonstad |
| collection | DOAJ |
| description | Using standard digital cameras in combination with deep learning (DL) for pose estimation is promising for the in-home and independent use of exercise games (exergames). We need to investigate to what extent such DL-based systems can provide satisfying accuracy on exergame relevant measures. Our study assesses temporal variation (i.e., variability) in body segment lengths, while using a Deep Learning image processing tool (DeepLabCut, DLC) on two-dimensional (2D) video. This variability is then compared with a gold-standard, marker-based three-dimensional Motion Capturing system (3DMoCap, Qualisys AB), and a 3D RGB-depth camera system (Kinect V2, Microsoft Inc). Simultaneous data were collected from all three systems, while participants (N = 12) played a custom balance training exergame. The pose estimation DLC-model is pre-trained on a large-scale dataset (ImageNet) and optimized with context-specific pose annotated images. Wilcoxon’s signed-rank test was performed in order to assess the statistical significance of the differences in variability between systems. The results showed that the DLC method performs comparably to the Kinect and, in some segments, even to the 3DMoCap gold standard system with regard to variability. These results are promising for making exergames more accessible and easier to use, thereby increasing their availability for in-home exercise. |
| first_indexed | 2024-03-10T14:19:53Z |
| format | Article |
| id | doaj.art-e6f3ea23cc2342af9a71c10be3d5e17b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T14:19:53Z |
| publishDate | 2020-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-e6f3ea23cc2342af9a71c10be3d5e17b2023-11-20T23:30:40ZengMDPI AGSensors1424-82202020-12-012023694010.3390/s20236940Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance TrainingElise Klæbo Vonstad0Xiaomeng Su1Beatrix Vereijken2Kerstin Bach3Jan Harald Nilsen4Department of Computer Science, Norwegian University of Science and Technology, 7034 Trondheim, NorwayDepartment of Computer Science, Norwegian University of Science and Technology, 7034 Trondheim, NorwayDepartment of Neuromedicine and Movement Science, Norwegian University of Science and Technology, 7030 Trondheim, NorwayDepartment of Computer Science, Norwegian University of Science and Technology, 7034 Trondheim, NorwayDepartment of Computer Science, Norwegian University of Science and Technology, 7034 Trondheim, NorwayUsing standard digital cameras in combination with deep learning (DL) for pose estimation is promising for the in-home and independent use of exercise games (exergames). We need to investigate to what extent such DL-based systems can provide satisfying accuracy on exergame relevant measures. Our study assesses temporal variation (i.e., variability) in body segment lengths, while using a Deep Learning image processing tool (DeepLabCut, DLC) on two-dimensional (2D) video. This variability is then compared with a gold-standard, marker-based three-dimensional Motion Capturing system (3DMoCap, Qualisys AB), and a 3D RGB-depth camera system (Kinect V2, Microsoft Inc). Simultaneous data were collected from all three systems, while participants (N = 12) played a custom balance training exergame. The pose estimation DLC-model is pre-trained on a large-scale dataset (ImageNet) and optimized with context-specific pose annotated images. Wilcoxon’s signed-rank test was performed in order to assess the statistical significance of the differences in variability between systems. The results showed that the DLC method performs comparably to the Kinect and, in some segments, even to the 3DMoCap gold standard system with regard to variability. These results are promising for making exergames more accessible and easier to use, thereby increasing their availability for in-home exercise.https://www.mdpi.com/1424-8220/20/23/6940motion captureimage analysismarkerless motion captureexergamingsegment lengthskinect |
| spellingShingle | Elise Klæbo Vonstad Xiaomeng Su Beatrix Vereijken Kerstin Bach Jan Harald Nilsen Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training Sensors motion capture image analysis markerless motion capture exergaming segment lengths kinect |
| title | Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training |
| title_full | Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training |
| title_fullStr | Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training |
| title_full_unstemmed | Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training |
| title_short | Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training |
| title_sort | comparison of a deep learning based pose estimation system to marker based and kinect systems in exergaming for balance training |
| topic | motion capture image analysis markerless motion capture exergaming segment lengths kinect |
| url | https://www.mdpi.com/1424-8220/20/23/6940 |
| work_keys_str_mv | AT eliseklæbovonstad comparisonofadeeplearningbasedposeestimationsystemtomarkerbasedandkinectsystemsinexergamingforbalancetraining AT xiaomengsu comparisonofadeeplearningbasedposeestimationsystemtomarkerbasedandkinectsystemsinexergamingforbalancetraining AT beatrixvereijken comparisonofadeeplearningbasedposeestimationsystemtomarkerbasedandkinectsystemsinexergamingforbalancetraining AT kerstinbach comparisonofadeeplearningbasedposeestimationsystemtomarkerbasedandkinectsystemsinexergamingforbalancetraining AT janharaldnilsen comparisonofadeeplearningbasedposeestimationsystemtomarkerbasedandkinectsystemsinexergamingforbalancetraining |