Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test
Most East Asian rehabilitation centers offer chopsticks manipulation tests (CMT). In addition to impaired hand function, approximately two-thirds of stroke survivors have visual impairment related to eye movement. This article investigates the significance of combining finger joint angle estimation...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9867997/ |
| _version_ | 1798001140958232576 |
|---|---|
| author | Adnan Rachmat Anom Besari Azhar Aulia Saputra Wei Hong Chin Kurnianingsih Naoyuki Kubota |
| author_facet | Adnan Rachmat Anom Besari Azhar Aulia Saputra Wei Hong Chin Kurnianingsih Naoyuki Kubota |
| author_sort | Adnan Rachmat Anom Besari |
| collection | DOAJ |
| description | Most East Asian rehabilitation centers offer chopsticks manipulation tests (CMT). In addition to impaired hand function, approximately two-thirds of stroke survivors have visual impairment related to eye movement. This article investigates the significance of combining finger joint angle estimation and a visual attention measurement in CMT. We present a multiscopic framework that consists of microscopic, mesoscopic, and macroscopic levels. We develop a feature extraction technique to build the finger kinematic model at the microscopic level. At the mesoscopic level, we propose an active perception ability to detect the position and geometry of the finger on the chopsticks. The proposed framework estimates the proximal interphalangeal (PIP) joint angle on the index finger during CMT using fully connected cascade neural networks (FCC-NN). At the macroscopic level, we implement a cognitive ability by measuring visual attention during CMT. We further evaluate the proposed framework with a conventional test that counts the number of peanuts (NP) which are moved from one bowl to another using chopsticks within a particular time frame. We introduce three evaluation indices, namely joint angle estimation movement (JAEM), chopstick attention movement (CAM), and chopstick tip movement (CTM), by detecting the local minima and maxima of the time series data. According to the experiment results, the velocity of these three evaluation indices could indicate improvement in hand and eye function during CMT. We expect this study to benefit therapists and researchers by providing valuable information that is not accessible in the clinic. Code and datasets are available online at <uri>https://github.com/anom-tmu/cmt-attention</uri>. |
| first_indexed | 2024-04-11T11:31:18Z |
| format | Article |
| id | doaj.art-41d84c118eba42718aaffcf144a1e6a9 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T11:31:18Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-41d84c118eba42718aaffcf144a1e6a92022-12-22T04:26:06ZengIEEEIEEE Access2169-35362022-01-0110913169133110.1109/ACCESS.2022.32018949867997Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation TestAdnan Rachmat Anom Besari0https://orcid.org/0000-0001-5916-5019Azhar Aulia Saputra1Wei Hong Chin2https://orcid.org/0000-0002-8592-9315 Kurnianingsih3https://orcid.org/0000-0001-7339-7449Naoyuki Kubota4https://orcid.org/0000-0001-8829-037XGraduate School of Systems Design, Tokyo Metropolitan University, Tokyo, JapanGraduate School of Systems Design, Tokyo Metropolitan University, Tokyo, JapanGraduate School of Systems Design, Tokyo Metropolitan University, Tokyo, JapanDepartment of Electrical Engineering, Politeknik Negeri Semarang, Semarang, IndonesiaGraduate School of Systems Design, Tokyo Metropolitan University, Tokyo, JapanMost East Asian rehabilitation centers offer chopsticks manipulation tests (CMT). In addition to impaired hand function, approximately two-thirds of stroke survivors have visual impairment related to eye movement. This article investigates the significance of combining finger joint angle estimation and a visual attention measurement in CMT. We present a multiscopic framework that consists of microscopic, mesoscopic, and macroscopic levels. We develop a feature extraction technique to build the finger kinematic model at the microscopic level. At the mesoscopic level, we propose an active perception ability to detect the position and geometry of the finger on the chopsticks. The proposed framework estimates the proximal interphalangeal (PIP) joint angle on the index finger during CMT using fully connected cascade neural networks (FCC-NN). At the macroscopic level, we implement a cognitive ability by measuring visual attention during CMT. We further evaluate the proposed framework with a conventional test that counts the number of peanuts (NP) which are moved from one bowl to another using chopsticks within a particular time frame. We introduce three evaluation indices, namely joint angle estimation movement (JAEM), chopstick attention movement (CAM), and chopstick tip movement (CTM), by detecting the local minima and maxima of the time series data. According to the experiment results, the velocity of these three evaluation indices could indicate improvement in hand and eye function during CMT. We expect this study to benefit therapists and researchers by providing valuable information that is not accessible in the clinic. Code and datasets are available online at <uri>https://github.com/anom-tmu/cmt-attention</uri>.https://ieeexplore.ieee.org/document/9867997/Hand-eye interactionrehabilitation evaluationfirst-person visioneye-gazeeye-tracking |
| spellingShingle | Adnan Rachmat Anom Besari Azhar Aulia Saputra Wei Hong Chin Kurnianingsih Naoyuki Kubota Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test IEEE Access Hand-eye interaction rehabilitation evaluation first-person vision eye-gaze eye-tracking |
| title | Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test |
| title_full | Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test |
| title_fullStr | Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test |
| title_full_unstemmed | Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test |
| title_short | Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test |
| title_sort | finger joint angle estimation with visual attention for rehabilitation support a case study of the chopsticks manipulation test |
| topic | Hand-eye interaction rehabilitation evaluation first-person vision eye-gaze eye-tracking |
| url | https://ieeexplore.ieee.org/document/9867997/ |
| work_keys_str_mv | AT adnanrachmatanombesari fingerjointangleestimationwithvisualattentionforrehabilitationsupportacasestudyofthechopsticksmanipulationtest AT azharauliasaputra fingerjointangleestimationwithvisualattentionforrehabilitationsupportacasestudyofthechopsticksmanipulationtest AT weihongchin fingerjointangleestimationwithvisualattentionforrehabilitationsupportacasestudyofthechopsticksmanipulationtest AT kurnianingsih fingerjointangleestimationwithvisualattentionforrehabilitationsupportacasestudyofthechopsticksmanipulationtest AT naoyukikubota fingerjointangleestimationwithvisualattentionforrehabilitationsupportacasestudyofthechopsticksmanipulationtest |