Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study
Recently, virtua reality (VR) has been widely utilized with rehabilitation to promote user engagement, which has been shown to induce brain plasticity. In this study, we developed a VR-based hand rehabilitation system consisting of a personalized gesture-controlled rhythm game with vibrotactile feed...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10239483/ |
| _version_ | 1827804812299730944 |
|---|---|
| author | Sungjin Bae Hyung-Soon Park |
| author_facet | Sungjin Bae Hyung-Soon Park |
| author_sort | Sungjin Bae |
| collection | DOAJ |
| description | Recently, virtua reality (VR) has been widely utilized with rehabilitation to promote user engagement, which has been shown to induce brain plasticity. In this study, we developed a VR-based hand rehabilitation system consisting of a personalized gesture-controlled rhythm game with vibrotactile feedback and investigated the cortical activation pattern induced by our system using functional near-infrared spectroscopy (fNIRS). Our system provides vibrotactile feedback as the user matches their hand gestures to VR targets customized to their pre-recorded hand gestures. Cortical activation was measured via fNIRS during 420 seconds of alternating gameplay and rest in 11 healthy subjects and one stroke survivor. Regions of interest (ROI) were the prefrontal cortex (PFC), the premotor cortex & the supplementary motor area (PMC&SMA), the primary sensorimotor cortex (SM1), and the somatosensory association cortex (SAC). The mean success rate of gesture matching among healthy subjects was 90 % with a standard deviation of 10.7 %, and the success rate of the stroke survivor was 79.6 %. The averaged cortical activation map for the 11 healthy subjects and the individual cortical activation map for the single stroke survivor showed increased hemodynamic responses of oxygenated hemoglobin (HbO) during the VR-based hand rehabilitation compared to the resting condition. Paired t-test analysis demonstrated a significant increase in HbO activation values in 19 out of 51 channels, corresponding to all ROIs except the left PFC and PMC&SMA, which exhibited high subject variability. The experimental results indicate that the proposed system successfully activated brain areas related to motor planning/execution, multisensory integration, and attention. |
| first_indexed | 2024-03-11T21:16:49Z |
| format | Article |
| id | doaj.art-4f48bf52370d4a4bbbd5519ee9f19827 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-11T21:16:49Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-4f48bf52370d4a4bbbd5519ee9f198272023-09-28T23:00:05ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01313732374310.1109/TNSRE.2023.331233610239483Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot StudySungjin Bae0https://orcid.org/0000-0003-1732-8507Hyung-Soon Park1https://orcid.org/0000-0003-4274-7420Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaDepartment of Mechanical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaRecently, virtua reality (VR) has been widely utilized with rehabilitation to promote user engagement, which has been shown to induce brain plasticity. In this study, we developed a VR-based hand rehabilitation system consisting of a personalized gesture-controlled rhythm game with vibrotactile feedback and investigated the cortical activation pattern induced by our system using functional near-infrared spectroscopy (fNIRS). Our system provides vibrotactile feedback as the user matches their hand gestures to VR targets customized to their pre-recorded hand gestures. Cortical activation was measured via fNIRS during 420 seconds of alternating gameplay and rest in 11 healthy subjects and one stroke survivor. Regions of interest (ROI) were the prefrontal cortex (PFC), the premotor cortex & the supplementary motor area (PMC&SMA), the primary sensorimotor cortex (SM1), and the somatosensory association cortex (SAC). The mean success rate of gesture matching among healthy subjects was 90 % with a standard deviation of 10.7 %, and the success rate of the stroke survivor was 79.6 %. The averaged cortical activation map for the 11 healthy subjects and the individual cortical activation map for the single stroke survivor showed increased hemodynamic responses of oxygenated hemoglobin (HbO) during the VR-based hand rehabilitation compared to the resting condition. Paired t-test analysis demonstrated a significant increase in HbO activation values in 19 out of 51 channels, corresponding to all ROIs except the left PFC and PMC&SMA, which exhibited high subject variability. The experimental results indicate that the proposed system successfully activated brain areas related to motor planning/execution, multisensory integration, and attention.https://ieeexplore.ieee.org/document/10239483/Functional near-infrared spectroscopygesture recognitionhand rehabilitation systemsvibrotactile feedbackvirtual reality |
| spellingShingle | Sungjin Bae Hyung-Soon Park Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study IEEE Transactions on Neural Systems and Rehabilitation Engineering Functional near-infrared spectroscopy gesture recognition hand rehabilitation systems vibrotactile feedback virtual reality |
| title | Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study |
| title_full | Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study |
| title_fullStr | Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study |
| title_full_unstemmed | Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study |
| title_short | Development of Immersive Virtual Reality-Based Hand Rehabilitation System Using a Gesture-Controlled Rhythm Game With Vibrotactile Feedback: An fNIRS Pilot Study |
| title_sort | development of immersive virtual reality based hand rehabilitation system using a gesture controlled rhythm game with vibrotactile feedback an fnirs pilot study |
| topic | Functional near-infrared spectroscopy gesture recognition hand rehabilitation systems vibrotactile feedback virtual reality |
| url | https://ieeexplore.ieee.org/document/10239483/ |
| work_keys_str_mv | AT sungjinbae developmentofimmersivevirtualrealitybasedhandrehabilitationsystemusingagesturecontrolledrhythmgamewithvibrotactilefeedbackanfnirspilotstudy AT hyungsoonpark developmentofimmersivevirtualrealitybasedhandrehabilitationsystemusingagesturecontrolledrhythmgamewithvibrotactilefeedbackanfnirspilotstudy |