Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones
Abstract The emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2021-07-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-021-92776-x |
| _version_ | 1818683256306925568 |
|---|---|
| author | Andrew P. Creagh Florian Lipsmeier Michael Lindemann Maarten De Vos |
| author_facet | Andrew P. Creagh Florian Lipsmeier Michael Lindemann Maarten De Vos |
| author_sort | Andrew P. Creagh |
| collection | DOAJ |
| description | Abstract The emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural Networks (DCNN) may capture a richer representation of healthy and MS-related ambulatory characteristics from the raw smartphone-based inertial sensor data than standard feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework leveraged the ambulatory information learned on human activity recognition (HAR) tasks collected from wearable smartphone sensor data. It was demonstrated that fine-tuning TL DCNN HAR models towards MS disease recognition tasks outperformed previous Support Vector Machine (SVM) feature-based methods, as well as DCNN models trained end-to-end, by upwards of 8–15%. A lack of transparency of “black-box” deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus people with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions. |
| first_indexed | 2024-12-17T10:31:51Z |
| format | Article |
| id | doaj.art-ef240cad88a843779cae0ef61ba5cb57 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-12-17T10:31:51Z |
| publishDate | 2021-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-ef240cad88a843779cae0ef61ba5cb572022-12-21T21:52:30ZengNature PortfolioScientific Reports2045-23222021-07-0111111410.1038/s41598-021-92776-xInterpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphonesAndrew P. Creagh0Florian Lipsmeier1Michael Lindemann2Maarten De Vos3Institute of Biomedical Engineering, Department of Engineering Science, University of OxfordF. Hoffmann-La RocheF. Hoffmann-La RocheInstitute of Biomedical Engineering, Department of Engineering Science, University of OxfordAbstract The emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural Networks (DCNN) may capture a richer representation of healthy and MS-related ambulatory characteristics from the raw smartphone-based inertial sensor data than standard feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework leveraged the ambulatory information learned on human activity recognition (HAR) tasks collected from wearable smartphone sensor data. It was demonstrated that fine-tuning TL DCNN HAR models towards MS disease recognition tasks outperformed previous Support Vector Machine (SVM) feature-based methods, as well as DCNN models trained end-to-end, by upwards of 8–15%. A lack of transparency of “black-box” deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus people with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions.https://doi.org/10.1038/s41598-021-92776-x |
| spellingShingle | Andrew P. Creagh Florian Lipsmeier Michael Lindemann Maarten De Vos Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones Scientific Reports |
| title | Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones |
| title_full | Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones |
| title_fullStr | Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones |
| title_full_unstemmed | Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones |
| title_short | Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones |
| title_sort | interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones |
| url | https://doi.org/10.1038/s41598-021-92776-x |
| work_keys_str_mv | AT andrewpcreagh interpretabledeeplearningfortheremotecharacterisationofambulationinmultiplesclerosisusingsmartphones AT florianlipsmeier interpretabledeeplearningfortheremotecharacterisationofambulationinmultiplesclerosisusingsmartphones AT michaellindemann interpretabledeeplearningfortheremotecharacterisationofambulationinmultiplesclerosisusingsmartphones AT maartendevos interpretabledeeplearningfortheremotecharacterisationofambulationinmultiplesclerosisusingsmartphones |