Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation
Gait impairment in Parkinson’s Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10387461/ |
| _version_ | 1797348776074018816 |
|---|---|
| author | Haoyu Tian Haiyun Li Wenjing Jiang Xin Ma Xiang Li Hanbo Wu Yibin Li |
| author_facet | Haoyu Tian Haiyun Li Wenjing Jiang Xin Ma Xiang Li Hanbo Wu Yibin Li |
| author_sort | Haoyu Tian |
| collection | DOAJ |
| description | Gait impairment in Parkinson’s Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24–50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset. |
| first_indexed | 2024-03-08T12:10:54Z |
| format | Article |
| id | doaj.art-a387d588c37f4c3baced8e051d0d6953 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-08T12:10:54Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-a387d588c37f4c3baced8e051d0d69532024-01-23T00:00:09ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102024-01-013241242110.1109/TNSRE.2024.335200410387461Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score EstimationHaoyu Tian0https://orcid.org/0000-0003-3789-2084Haiyun Li1https://orcid.org/0000-0001-6803-4117Wenjing Jiang2https://orcid.org/0000-0002-1559-9744Xin Ma3https://orcid.org/0000-0003-4402-1957Xiang Li4https://orcid.org/0000-0003-1529-7057Hanbo Wu5https://orcid.org/0000-0002-7924-5072Yibin Li6https://orcid.org/0000-0002-5906-5074Center of Robotics School of Control Science and Engineering, Shandong University, Jinan, ChinaQilu Hospital of Shandong University, Jinan, ChinaQilu Hospital of Shandong University, Jinan, ChinaCenter of Robotics School of Control Science and Engineering, Shandong University, Jinan, ChinaCenter of Robotics School of Control Science and Engineering, Shandong University, Jinan, ChinaCenter of Robotics School of Control Science and Engineering, Shandong University, Jinan, ChinaCenter of Robotics School of Control Science and Engineering, Shandong University, Jinan, ChinaGait impairment in Parkinson’s Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24–50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset.https://ieeexplore.ieee.org/document/10387461/Parkinsonian gaitquantitative assessmentgraph convolutional networkskeleton-based data |
| spellingShingle | Haoyu Tian Haiyun Li Wenjing Jiang Xin Ma Xiang Li Hanbo Wu Yibin Li Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation IEEE Transactions on Neural Systems and Rehabilitation Engineering Parkinsonian gait quantitative assessment graph convolutional network skeleton-based data |
| title | Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation |
| title_full | Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation |
| title_fullStr | Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation |
| title_full_unstemmed | Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation |
| title_short | Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation |
| title_sort | cross spatiotemporal graph convolution networks for skeleton based parkinsonian gait mds updrs score estimation |
| topic | Parkinsonian gait quantitative assessment graph convolutional network skeleton-based data |
| url | https://ieeexplore.ieee.org/document/10387461/ |
| work_keys_str_mv | AT haoyutian crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation AT haiyunli crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation AT wenjingjiang crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation AT xinma crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation AT xiangli crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation AT hanbowu crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation AT yibinli crossspatiotemporalgraphconvolutionnetworksforskeletonbasedparkinsoniangaitmdsupdrsscoreestimation |