Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video
Markerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the m...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-01-01
|
| Series: | Frontiers in Bioengineering and Biotechnology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fbioe.2023.1335251/full |
| _version_ | 1827386855926005760 |
|---|---|
| author | Rui Hu Rui Hu Yanan Diao Yanan Diao Yingchi Wang Gaoqiang Li Rong He Yunkun Ning Nan Lou Guanglin Li Guoru Zhao |
| author_facet | Rui Hu Rui Hu Yanan Diao Yanan Diao Yingchi Wang Gaoqiang Li Rong He Yunkun Ning Nan Lou Guanglin Li Guoru Zhao |
| author_sort | Rui Hu |
| collection | DOAJ |
| description | Markerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the more promising single-view technology has defects such as low accuracy and reliability, which seriously limits clinical application. This study proposes a high-resolution graph convolutional multilayer perception (HGcnMLP) human 3D pose estimation framework for smartphone monocular videos and estimates 15 healthy adults and 12 patients with musculoskeletal disorders (sarcopenia and osteoarthritis) gait spatiotemporal, knee angle, and center-of-mass (COM) velocity parameters, etc., and compared with the VICON gold standard system. The results show that most of the calculated parameters have excellent reliability (VICON, ICC (2, k): 0.853–0.982; Phone, ICC (2, k): 0.839–0.975) and validity (Pearson r: 0.808–0.978, p<0.05). In addition, the proposed system can better evaluate human gait balance ability, and the K-means++ clustering algorithm can successfully distinguish patients into different recovery level groups. This study verifies the potential of a single smartphone video for 3D human pose estimation for rehabilitation auxiliary diagnosis and balance level recognition, and is an effective attempt at the clinical application of emerging computer vision technology. In the future, it is hoped that the corresponding smartphone program will be developed to provide a low-cost, effective, and simple new tool for remote monitoring and rehabilitation assessment of patients. |
| first_indexed | 2024-03-08T15:51:40Z |
| format | Article |
| id | doaj.art-6675930869894dcc92fba5527c28ff6b |
| institution | Directory Open Access Journal |
| issn | 2296-4185 |
| language | English |
| last_indexed | 2024-03-08T15:51:40Z |
| publishDate | 2024-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Bioengineering and Biotechnology |
| spelling | doaj.art-6675930869894dcc92fba5527c28ff6b2024-01-09T04:25:35ZengFrontiers Media S.A.Frontiers in Bioengineering and Biotechnology2296-41852024-01-011110.3389/fbioe.2023.13352511335251Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular videoRui Hu0Rui Hu1Yanan Diao2Yanan Diao3Yingchi Wang4Gaoqiang Li5Rong He6Yunkun Ning7Nan Lou8Guanglin Li9Guoru Zhao10CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaShenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, ChinaCAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaShenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, ChinaCAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaDepartment of Orthopedic and Rehabilitation Center, University of Hong Kong–Shenzhen Hospital, Shenzhen, ChinaDepartment of Orthopedic and Rehabilitation Center, University of Hong Kong–Shenzhen Hospital, Shenzhen, ChinaCAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaDepartment of Orthopedic and Rehabilitation Center, University of Hong Kong–Shenzhen Hospital, Shenzhen, ChinaCAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaCAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaMarkerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the more promising single-view technology has defects such as low accuracy and reliability, which seriously limits clinical application. This study proposes a high-resolution graph convolutional multilayer perception (HGcnMLP) human 3D pose estimation framework for smartphone monocular videos and estimates 15 healthy adults and 12 patients with musculoskeletal disorders (sarcopenia and osteoarthritis) gait spatiotemporal, knee angle, and center-of-mass (COM) velocity parameters, etc., and compared with the VICON gold standard system. The results show that most of the calculated parameters have excellent reliability (VICON, ICC (2, k): 0.853–0.982; Phone, ICC (2, k): 0.839–0.975) and validity (Pearson r: 0.808–0.978, p<0.05). In addition, the proposed system can better evaluate human gait balance ability, and the K-means++ clustering algorithm can successfully distinguish patients into different recovery level groups. This study verifies the potential of a single smartphone video for 3D human pose estimation for rehabilitation auxiliary diagnosis and balance level recognition, and is an effective attempt at the clinical application of emerging computer vision technology. In the future, it is hoped that the corresponding smartphone program will be developed to provide a low-cost, effective, and simple new tool for remote monitoring and rehabilitation assessment of patients.https://www.frontiersin.org/articles/10.3389/fbioe.2023.1335251/fullmarkerless pose estimationrehabilitation assessmenthigh-resolution graph convolutional multilayer perception (HGcnMLP)smartphone monocular videomusculoskeletal disorders |
| spellingShingle | Rui Hu Rui Hu Yanan Diao Yanan Diao Yingchi Wang Gaoqiang Li Rong He Yunkun Ning Nan Lou Guanglin Li Guoru Zhao Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video Frontiers in Bioengineering and Biotechnology markerless pose estimation rehabilitation assessment high-resolution graph convolutional multilayer perception (HGcnMLP) smartphone monocular video musculoskeletal disorders |
| title | Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video |
| title_full | Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video |
| title_fullStr | Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video |
| title_full_unstemmed | Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video |
| title_short | Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video |
| title_sort | effective evaluation of hgcnmlp method for markerless 3d pose estimation of musculoskeletal diseases patients based on smartphone monocular video |
| topic | markerless pose estimation rehabilitation assessment high-resolution graph convolutional multilayer perception (HGcnMLP) smartphone monocular video musculoskeletal disorders |
| url | https://www.frontiersin.org/articles/10.3389/fbioe.2023.1335251/full |
| work_keys_str_mv | AT ruihu effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT ruihu effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT yanandiao effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT yanandiao effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT yingchiwang effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT gaoqiangli effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT ronghe effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT yunkunning effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT nanlou effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT guanglinli effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo AT guoruzhao effectiveevaluationofhgcnmlpmethodformarkerless3dposeestimationofmusculoskeletaldiseasespatientsbasedonsmartphonemonocularvideo |