Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management
Post-stroke care encounters challenges, including high cost, lack of professionals, and insufficient rehabilitation state evaluation. Computer technology can alleviate these issues, as it allows health care professionals (HCP) to quantify the workload and thus enhance rehabilitation care quality. In...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9440405/ |
| _version_ | 1818618071873486848 |
|---|---|
| author | Yao Tong Zhenxiang Zhang Gang Chen Xin Li Hang Yan Mengya Xu Beilei Lin |
| author_facet | Yao Tong Zhenxiang Zhang Gang Chen Xin Li Hang Yan Mengya Xu Beilei Lin |
| author_sort | Yao Tong |
| collection | DOAJ |
| description | Post-stroke care encounters challenges, including high cost, lack of professionals, and insufficient rehabilitation state evaluation. Computer technology can alleviate these issues, as it allows health care professionals (HCP) to quantify the workload and thus enhance rehabilitation care quality. In this paper, a novel multi-model fusion method, terms as pose dual-stream network (PDSN), is devised, aiming to test the feasibility of monitoring the training actions of rehabilitating stroke patients in care management. In particular, this deep-learning-based algorithm combines human pose estimation and dual-stream networks in an innovative way. We utilize an improved OpenPose to estimate human pose from videos obtained by the low-cost monocular camera. In dual-stream networks, the spatial and motion streams are flexibly integrated. The spatial stream network combines the Gated Recurrent Unit (GRU) and attention mechanism to extract spatiotemporal data, while the motion stream network is composed of improved multi-layer 1D Convolutional Neural Networks (CNN), which enhanced by causal and dilated convolution skillfully. Additionally, an adaptive weight fusion strategy is used to fuse the two networks for the final action classification. Results show high accuracy on two public datasets and a dataset created by us, which validate the superiority and feasibility of our method. |
| first_indexed | 2024-12-16T17:15:46Z |
| format | Article |
| id | doaj.art-1699c61be3bc4d7c80694622f70691d0 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T17:15:46Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1699c61be3bc4d7c80694622f70691d02022-12-21T22:23:17ZengIEEEIEEE Access2169-35362021-01-019781747818710.1109/ACCESS.2021.30836689440405Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care ManagementYao Tong0https://orcid.org/0000-0002-7885-3922Zhenxiang Zhang1Gang Chen2https://orcid.org/0000-0002-6290-7041Xin Li3https://orcid.org/0000-0003-0576-3181Hang Yan4https://orcid.org/0000-0003-3912-0168Mengya Xu5https://orcid.org/0000-0001-5336-8237Beilei Lin6https://orcid.org/0000-0002-6502-7402School of Nursing and Health, Zhengzhou University, Zhengzhou, ChinaSchool of Nursing and Health, Zhengzhou University, Zhengzhou, ChinaSchool of Information Engineering, Zhengzhou University, Zhengzhou, ChinaCollege of Computer and Information, Hohai University, Nanjing, ChinaSchool of Information Engineering, Zhengzhou University, Zhengzhou, ChinaDepartment of Rehabilitation, The Second Affiliated Hospital of Zhengzhou University, Zhengzhou, ChinaSchool of Nursing and Health, Zhengzhou University, Zhengzhou, ChinaPost-stroke care encounters challenges, including high cost, lack of professionals, and insufficient rehabilitation state evaluation. Computer technology can alleviate these issues, as it allows health care professionals (HCP) to quantify the workload and thus enhance rehabilitation care quality. In this paper, a novel multi-model fusion method, terms as pose dual-stream network (PDSN), is devised, aiming to test the feasibility of monitoring the training actions of rehabilitating stroke patients in care management. In particular, this deep-learning-based algorithm combines human pose estimation and dual-stream networks in an innovative way. We utilize an improved OpenPose to estimate human pose from videos obtained by the low-cost monocular camera. In dual-stream networks, the spatial and motion streams are flexibly integrated. The spatial stream network combines the Gated Recurrent Unit (GRU) and attention mechanism to extract spatiotemporal data, while the motion stream network is composed of improved multi-layer 1D Convolutional Neural Networks (CNN), which enhanced by causal and dilated convolution skillfully. Additionally, an adaptive weight fusion strategy is used to fuse the two networks for the final action classification. Results show high accuracy on two public datasets and a dataset created by us, which validate the superiority and feasibility of our method.https://ieeexplore.ieee.org/document/9440405/Stroke rehabilitationaction recognitiondeep learningpatient care managementmulti-model fusion |
| spellingShingle | Yao Tong Zhenxiang Zhang Gang Chen Xin Li Hang Yan Mengya Xu Beilei Lin Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management IEEE Access Stroke rehabilitation action recognition deep learning patient care management multi-model fusion |
| title | Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management |
| title_full | Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management |
| title_fullStr | Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management |
| title_full_unstemmed | Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management |
| title_short | Testing the Feasibility of a Multi-Model Fusion Method for Monitoring the Action of Rehabilitating Stroke Patients in Care Management |
| title_sort | testing the feasibility of a multi model fusion method for monitoring the action of rehabilitating stroke patients in care management |
| topic | Stroke rehabilitation action recognition deep learning patient care management multi-model fusion |
| url | https://ieeexplore.ieee.org/document/9440405/ |
| work_keys_str_mv | AT yaotong testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement AT zhenxiangzhang testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement AT gangchen testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement AT xinli testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement AT hangyan testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement AT mengyaxu testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement AT beileilin testingthefeasibilityofamultimodelfusionmethodformonitoringtheactionofrehabilitatingstrokepatientsincaremanagement |