Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow
We seek to predict knee and ankle motion using wearable sensors. These predictions could serve as target trajectories for a lower limb prosthesis. In this manuscript, we investigate the use of egocentric vision for improving performance over kinematic wearable motion capture. We present an out-of-th...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9729197/ |
| _version_ | 1797805135434350592 |
|---|---|
| author | Abhishek Sharma Eric Rombokas |
| author_facet | Abhishek Sharma Eric Rombokas |
| author_sort | Abhishek Sharma |
| collection | DOAJ |
| description | We seek to predict knee and ankle motion using wearable sensors. These predictions could serve as target trajectories for a lower limb prosthesis. In this manuscript, we investigate the use of egocentric vision for improving performance over kinematic wearable motion capture. We present an out-of-the-lab dataset of 23 healthy subjects navigating public classrooms, a large atrium, and stairs for a total of almost 12 hours of recording. The prediction task is difficult because the movements include avoiding obstacles, other people, idiosyncratic movements such as traversing doors, and individual choices in selecting the future path. We demonstrate that using vision improves the quality of the predicted knee and ankle trajectories, especially in congested spaces and when the visual environment provides information that does not appear simply in the movements of the body. Overall, including vision results in 7.9% and 7.0% improvement in root mean squared error of knee and ankle angle predictions respectively. The improvement in Pearson Correlation Coefficient for knee and ankle predictions is 1.5% and 12.3% respectively. We discuss particular moments where vision greatly improved, or failed to improve, the prediction performance. We also find that the benefits of vision can be enhanced with more data. Lastly, we discuss challenges of continuous estimation of gait in natural, out-of-the-lab datasets. |
| first_indexed | 2024-03-13T05:46:43Z |
| format | Article |
| id | doaj.art-184443604a9a456eb545145b621fad0c |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-13T05:46:43Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-184443604a9a456eb545145b621fad0c2023-06-13T20:08:14ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102022-01-013069970810.1109/TNSRE.2022.31568849729197Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical FlowAbhishek Sharma0https://orcid.org/0000-0001-6666-2179Eric Rombokas1https://orcid.org/0000-0001-8523-1913Department of Mechanical Engineering, University of Washington, Seattle, WA, USADepartment of Mechanical Engineering, University of Washington, Seattle, WA, USAWe seek to predict knee and ankle motion using wearable sensors. These predictions could serve as target trajectories for a lower limb prosthesis. In this manuscript, we investigate the use of egocentric vision for improving performance over kinematic wearable motion capture. We present an out-of-the-lab dataset of 23 healthy subjects navigating public classrooms, a large atrium, and stairs for a total of almost 12 hours of recording. The prediction task is difficult because the movements include avoiding obstacles, other people, idiosyncratic movements such as traversing doors, and individual choices in selecting the future path. We demonstrate that using vision improves the quality of the predicted knee and ankle trajectories, especially in congested spaces and when the visual environment provides information that does not appear simply in the movements of the body. Overall, including vision results in 7.9% and 7.0% improvement in root mean squared error of knee and ankle angle predictions respectively. The improvement in Pearson Correlation Coefficient for knee and ankle predictions is 1.5% and 12.3% respectively. We discuss particular moments where vision greatly improved, or failed to improve, the prediction performance. We also find that the benefits of vision can be enhanced with more data. Lastly, we discuss challenges of continuous estimation of gait in natural, out-of-the-lab datasets.https://ieeexplore.ieee.org/document/9729197/Egocentric visiondeep learninggait predictionprostheticsregression |
| spellingShingle | Abhishek Sharma Eric Rombokas Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow IEEE Transactions on Neural Systems and Rehabilitation Engineering Egocentric vision deep learning gait prediction prosthetics regression |
| title | Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow |
| title_full | Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow |
| title_fullStr | Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow |
| title_full_unstemmed | Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow |
| title_short | Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow |
| title_sort | improving imu based prediction of lower limb kinematics in natural environments using egocentric optical flow |
| topic | Egocentric vision deep learning gait prediction prosthetics regression |
| url | https://ieeexplore.ieee.org/document/9729197/ |
| work_keys_str_mv | AT abhisheksharma improvingimubasedpredictionoflowerlimbkinematicsinnaturalenvironmentsusingegocentricopticalflow AT ericrombokas improvingimubasedpredictionoflowerlimbkinematicsinnaturalenvironmentsusingegocentricopticalflow |