Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors
Aims: An objective technological solution for tracking adherence to at-home shoulder physiotherapy is important for improving patient engagement and rehabilitation outcomes, but remains a significant challenge. The aim of this research was to evaluate performance of machine-learning (ML) methodolog...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The British Editorial Society of Bone & Joint Surgery
2023-03-01
|
| Series: | Bone & Joint Research |
| Subjects: | |
| Online Access: | https://online.boneandjoint.org.uk/doi/epdf/10.1302/2046-3758.123.BJR-2022-0126.R1 |
| _version_ | 1797853294274543616 |
|---|---|
| author | Philip Boyer David Burns Cari Whyne |
| author_facet | Philip Boyer David Burns Cari Whyne |
| author_sort | Philip Boyer |
| collection | DOAJ |
| description | Aims: An objective technological solution for tracking adherence to at-home shoulder physiotherapy is important for improving patient engagement and rehabilitation outcomes, but remains a significant challenge. The aim of this research was to evaluate performance of machine-learning (ML) methodologies for detecting and classifying inertial data collected during in-clinic and at-home shoulder physiotherapy exercise. Methods: A smartwatch was used to collect inertial data from 42 patients performing shoulder physiotherapy exercises for rotator cuff injuries in both in-clinic and at-home settings. A two-stage ML approach was used to detect out-of-distribution (OOD) data (to remove non-exercise data) and subsequently for classification of exercises. We evaluated the performance impact of grouping exercises by motion type, inclusion of non-exercise data for algorithm training, and a patient-specific approach to exercise classification. Algorithm performance was evaluated using both in-clinic and at-home data. Results: The patient-specific approach with engineered features achieved the highest in-clinic performance for differentiating physiotherapy exercise from non-exercise activity (area under the receiver operating characteristic (AUROC) = 0.924). Including non-exercise data in algorithm training further improved classifier performance (random forest, AUROC = 0.985). The highest accuracy achieved for classifying individual in-clinic exercises was 0.903, using a patient-specific method with deep neural network model extracted features. Grouping exercises by motion type improved exercise classification. For at-home data, OOD detection yielded similar performance with the non-exercise data in the algorithm training (fully convolutional network AUROC = 0.919). Conclusion: Including non-exercise data in algorithm training improves detection of exercises. A patient-specific approach leveraging data from earlier patient-supervised sessions should be considered but is highly dependent on per-patient data quality. Cite this article: Bone Joint Res 2023;12(3):165–177. |
| first_indexed | 2024-04-09T19:48:16Z |
| format | Article |
| id | doaj.art-06db1c4dd8c649859abb48ede6b715a9 |
| institution | Directory Open Access Journal |
| issn | 2046-3758 |
| language | English |
| last_indexed | 2024-04-09T19:48:16Z |
| publishDate | 2023-03-01 |
| publisher | The British Editorial Society of Bone & Joint Surgery |
| record_format | Article |
| series | Bone & Joint Research |
| spelling | doaj.art-06db1c4dd8c649859abb48ede6b715a92023-04-03T12:02:00ZengThe British Editorial Society of Bone & Joint SurgeryBone & Joint Research2046-37582023-03-0112316517710.1302/2046-3758.123.BJR-2022-0126.R1Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensorsPhilip Boyer0David Burns1Cari Whyne2Institute of Biomedical Engineering, University of Toronto, Toronto, CanadaHarborview Medical Center, Seattle, Washington, USAInstitute of Biomedical Engineering, University of Toronto, Toronto, CanadaAims: An objective technological solution for tracking adherence to at-home shoulder physiotherapy is important for improving patient engagement and rehabilitation outcomes, but remains a significant challenge. The aim of this research was to evaluate performance of machine-learning (ML) methodologies for detecting and classifying inertial data collected during in-clinic and at-home shoulder physiotherapy exercise. Methods: A smartwatch was used to collect inertial data from 42 patients performing shoulder physiotherapy exercises for rotator cuff injuries in both in-clinic and at-home settings. A two-stage ML approach was used to detect out-of-distribution (OOD) data (to remove non-exercise data) and subsequently for classification of exercises. We evaluated the performance impact of grouping exercises by motion type, inclusion of non-exercise data for algorithm training, and a patient-specific approach to exercise classification. Algorithm performance was evaluated using both in-clinic and at-home data. Results: The patient-specific approach with engineered features achieved the highest in-clinic performance for differentiating physiotherapy exercise from non-exercise activity (area under the receiver operating characteristic (AUROC) = 0.924). Including non-exercise data in algorithm training further improved classifier performance (random forest, AUROC = 0.985). The highest accuracy achieved for classifying individual in-clinic exercises was 0.903, using a patient-specific method with deep neural network model extracted features. Grouping exercises by motion type improved exercise classification. For at-home data, OOD detection yielded similar performance with the non-exercise data in the algorithm training (fully convolutional network AUROC = 0.919). Conclusion: Including non-exercise data in algorithm training improves detection of exercises. A patient-specific approach leveraging data from earlier patient-supervised sessions should be considered but is highly dependent on per-patient data quality. Cite this article: Bone Joint Res 2023;12(3):165–177.https://online.boneandjoint.org.uk/doi/epdf/10.1302/2046-3758.123.BJR-2022-0126.R1physiotherapyphysical therapyrehabilitationinertial measurement unitsmachine learningphysiotherapyshoulderrotator cuff injuriesphysiotherapistsrotator cuffaccelerometerfull-thickness rotator cuff tearsvariancesstandard deviationflexion |
| spellingShingle | Philip Boyer David Burns Cari Whyne Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors Bone & Joint Research physiotherapy physical therapy rehabilitation inertial measurement units machine learning physiotherapy shoulder rotator cuff injuries physiotherapists rotator cuff accelerometer full-thickness rotator cuff tears variances standard deviation flexion |
| title | Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors |
| title_full | Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors |
| title_fullStr | Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors |
| title_full_unstemmed | Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors |
| title_short | Evaluation of at-home physiotherapy: machine-learning prediction with smart watch inertial sensors |
| title_sort | evaluation of at home physiotherapy machine learning prediction with smart watch inertial sensors |
| topic | physiotherapy physical therapy rehabilitation inertial measurement units machine learning physiotherapy shoulder rotator cuff injuries physiotherapists rotator cuff accelerometer full-thickness rotator cuff tears variances standard deviation flexion |
| url | https://online.boneandjoint.org.uk/doi/epdf/10.1302/2046-3758.123.BJR-2022-0126.R1 |
| work_keys_str_mv | AT philipboyer evaluationofathomephysiotherapymachinelearningpredictionwithsmartwatchinertialsensors AT davidburns evaluationofathomephysiotherapymachinelearningpredictionwithsmartwatchinertialsensors AT cariwhyne evaluationofathomephysiotherapymachinelearningpredictionwithsmartwatchinertialsensors |