Depth-aware pose estimation using deep learning for exoskeleton gait analysis
Abstract In rehabilitation medicine, real-time analysis of the gait for human wearing lower-limb exoskeleton rehabilitation robot during walking can effectively prevent patients from experiencing excessive and asymmetric gait during rehabilitation training, thereby avoiding falls or even secondary i...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-12-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-50207-z |
| _version_ | 1797377037753647104 |
|---|---|
| author | Yachun Wang Zhongcai Pei Chen Wang Zhiyong Tang |
| author_facet | Yachun Wang Zhongcai Pei Chen Wang Zhiyong Tang |
| author_sort | Yachun Wang |
| collection | DOAJ |
| description | Abstract In rehabilitation medicine, real-time analysis of the gait for human wearing lower-limb exoskeleton rehabilitation robot during walking can effectively prevent patients from experiencing excessive and asymmetric gait during rehabilitation training, thereby avoiding falls or even secondary injuries. To address the above situation, we propose a gait detection method based on computer vision for the real-time monitoring of gait during human–machine integrated walking. Specifically, we design a neural network model called GaitPoseNet, which is used for posture recognition in human–machine integrated walking. Using RGB images as input and depth features as output, regression of joint coordinates through depth estimation of implicit supervised networks. In addition, joint guidance strategy (JGS) is designed in the network framework. The degree of correlation between the various joints of the human body is used as a detection target to effectively overcome prediction difficulties due to partial joint occlusion during walking. Finally, a post processing algorithm is designed to describe patients’ walking motion by combining the pixel coordinates of each joint point and leg length. Our advantage is that we provide a non-contact measurement method with strong universality, and use depth estimation and JGS to improve measurement accuracy. Conducting experiments on the Walking Pose with Exoskeleton (WPE) Dataset shows that our method can reach 95.77% PCKs@0.1, 93.14% PCKs@0.08 and 3.55 ms runtime. Therefore our method achieves advanced performance considering both speed and accuracy. |
| first_indexed | 2024-03-08T19:47:15Z |
| format | Article |
| id | doaj.art-4de32989df844c4ab417917b46d2e5b7 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-08T19:47:15Z |
| publishDate | 2023-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-4de32989df844c4ab417917b46d2e5b72023-12-24T12:16:32ZengNature PortfolioScientific Reports2045-23222023-12-0113111610.1038/s41598-023-50207-zDepth-aware pose estimation using deep learning for exoskeleton gait analysisYachun Wang0Zhongcai Pei1Chen Wang2Zhiyong Tang3School of Automation Science and Electrical Engineering, Beihang UniversitySchool of Automation Science and Electrical Engineering, Beihang UniversitySchool of Automation Science and Electrical Engineering, Beihang UniversitySchool of Automation Science and Electrical Engineering, Beihang UniversityAbstract In rehabilitation medicine, real-time analysis of the gait for human wearing lower-limb exoskeleton rehabilitation robot during walking can effectively prevent patients from experiencing excessive and asymmetric gait during rehabilitation training, thereby avoiding falls or even secondary injuries. To address the above situation, we propose a gait detection method based on computer vision for the real-time monitoring of gait during human–machine integrated walking. Specifically, we design a neural network model called GaitPoseNet, which is used for posture recognition in human–machine integrated walking. Using RGB images as input and depth features as output, regression of joint coordinates through depth estimation of implicit supervised networks. In addition, joint guidance strategy (JGS) is designed in the network framework. The degree of correlation between the various joints of the human body is used as a detection target to effectively overcome prediction difficulties due to partial joint occlusion during walking. Finally, a post processing algorithm is designed to describe patients’ walking motion by combining the pixel coordinates of each joint point and leg length. Our advantage is that we provide a non-contact measurement method with strong universality, and use depth estimation and JGS to improve measurement accuracy. Conducting experiments on the Walking Pose with Exoskeleton (WPE) Dataset shows that our method can reach 95.77% PCKs@0.1, 93.14% PCKs@0.08 and 3.55 ms runtime. Therefore our method achieves advanced performance considering both speed and accuracy.https://doi.org/10.1038/s41598-023-50207-z |
| spellingShingle | Yachun Wang Zhongcai Pei Chen Wang Zhiyong Tang Depth-aware pose estimation using deep learning for exoskeleton gait analysis Scientific Reports |
| title | Depth-aware pose estimation using deep learning for exoskeleton gait analysis |
| title_full | Depth-aware pose estimation using deep learning for exoskeleton gait analysis |
| title_fullStr | Depth-aware pose estimation using deep learning for exoskeleton gait analysis |
| title_full_unstemmed | Depth-aware pose estimation using deep learning for exoskeleton gait analysis |
| title_short | Depth-aware pose estimation using deep learning for exoskeleton gait analysis |
| title_sort | depth aware pose estimation using deep learning for exoskeleton gait analysis |
| url | https://doi.org/10.1038/s41598-023-50207-z |
| work_keys_str_mv | AT yachunwang depthawareposeestimationusingdeeplearningforexoskeletongaitanalysis AT zhongcaipei depthawareposeestimationusingdeeplearningforexoskeletongaitanalysis AT chenwang depthawareposeestimationusingdeeplearningforexoskeletongaitanalysis AT zhiyongtang depthawareposeestimationusingdeeplearningforexoskeletongaitanalysis |