A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance
Automated exercise assessment is of great importance for patients under rehabilitation exercise who require professional guidance. Among the existing approaches, the skeleton-based assessment model that classifies the correctness of the exercise has attracted much attention due to its relative ease...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10144079/ |
| _version_ | 1797804906144333824 |
|---|---|
| author | Kaili Zheng Ji Wu Jialin Zhang Chenyi Guo |
| author_facet | Kaili Zheng Ji Wu Jialin Zhang Chenyi Guo |
| author_sort | Kaili Zheng |
| collection | DOAJ |
| description | Automated exercise assessment is of great importance for patients under rehabilitation exercise who require professional guidance. Among the existing approaches, the skeleton-based assessment model that classifies the correctness of the exercise has attracted much attention due to its relative ease of implementation and convenience in use. However, there are two problems with this approach. The first problem is its sensitivity to the orientation of the human skeleton. To solve this problem, we propose a novel rotation-invariant descriptor, the dot product matrix of the human skeleton, and prove mathematically that this descriptor discards only the orientation message that we do not expect while preserving all other useful information. Lack of feedback from the system is the second problem, because the exercisers do not know which parts of their exercises are incorrect. Therefore, we develop a visualization method for our system based on Gradient-Weighted Class Activation Mapping (Grad-CAM) and an quantitative metric called Overlap Ratio (OvR) to measure the quality of the visualization result. To demonstrate the effect of our method, we conduct experiments on two public datasets and a self-generated push-up dataset. The experimental results show that our rotation-invariant descriptor can achieve absolute robustness to orientation even under severe angle perturbations. In terms of accuracy and OvR, our method even outperforms previous works in most cases, indicating that the rotation-invariant descriptor helps the assessment model to extract more stable features. The visualization results are also informative to correct the movements; some examples are presented in this paper. The code of this paper and our push-up dataset are publicly available at <uri>https://github.com/Kelly510/RehabExerAssess</uri>. |
| first_indexed | 2024-03-13T05:44:05Z |
| format | Article |
| id | doaj.art-bb5137dc56f7429c915f87887817ec13 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-13T05:44:05Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-bb5137dc56f7429c915f87887817ec132023-06-13T23:00:17ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01312612262110.1109/TNSRE.2023.328267510144079A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation InvarianceKaili Zheng0https://orcid.org/0000-0001-7739-4659Ji Wu1https://orcid.org/0000-0001-6170-726XJialin Zhang2Chenyi Guo3https://orcid.org/0000-0002-8222-5179Department of Electronic Engineering and the Institute for Precision Medicine, Tsinghua University, Beijing, ChinaDepartment of Electronic Engineering and the Institute for Precision Medicine, Tsinghua University, Beijing, ChinaDepartment of Electronic Engineering and the Institute for Precision Medicine, Tsinghua University, Beijing, ChinaDepartment of Electronic Engineering and the Institute for Precision Medicine, Tsinghua University, Beijing, ChinaAutomated exercise assessment is of great importance for patients under rehabilitation exercise who require professional guidance. Among the existing approaches, the skeleton-based assessment model that classifies the correctness of the exercise has attracted much attention due to its relative ease of implementation and convenience in use. However, there are two problems with this approach. The first problem is its sensitivity to the orientation of the human skeleton. To solve this problem, we propose a novel rotation-invariant descriptor, the dot product matrix of the human skeleton, and prove mathematically that this descriptor discards only the orientation message that we do not expect while preserving all other useful information. Lack of feedback from the system is the second problem, because the exercisers do not know which parts of their exercises are incorrect. Therefore, we develop a visualization method for our system based on Gradient-Weighted Class Activation Mapping (Grad-CAM) and an quantitative metric called Overlap Ratio (OvR) to measure the quality of the visualization result. To demonstrate the effect of our method, we conduct experiments on two public datasets and a self-generated push-up dataset. The experimental results show that our rotation-invariant descriptor can achieve absolute robustness to orientation even under severe angle perturbations. In terms of accuracy and OvR, our method even outperforms previous works in most cases, indicating that the rotation-invariant descriptor helps the assessment model to extract more stable features. The visualization results are also informative to correct the movements; some examples are presented in this paper. The code of this paper and our push-up dataset are publicly available at <uri>https://github.com/Kelly510/RehabExerAssess</uri>.https://ieeexplore.ieee.org/document/10144079/Automated assessmentphysical rehabilitationrotation invariancespatial-temporal graph convolution |
| spellingShingle | Kaili Zheng Ji Wu Jialin Zhang Chenyi Guo A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance IEEE Transactions on Neural Systems and Rehabilitation Engineering Automated assessment physical rehabilitation rotation invariance spatial-temporal graph convolution |
| title | A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance |
| title_full | A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance |
| title_fullStr | A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance |
| title_full_unstemmed | A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance |
| title_short | A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance |
| title_sort | skeleton based rehabilitation exercise assessment system with rotation invariance |
| topic | Automated assessment physical rehabilitation rotation invariance spatial-temporal graph convolution |
| url | https://ieeexplore.ieee.org/document/10144079/ |
| work_keys_str_mv | AT kailizheng askeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT jiwu askeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT jialinzhang askeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT chenyiguo askeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT kailizheng skeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT jiwu skeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT jialinzhang skeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance AT chenyiguo skeletonbasedrehabilitationexerciseassessmentsystemwithrotationinvariance |