Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach
Japanese Sign Language (JSL) is vital for communication in Japan’s deaf and hard-of-hearing community. But probably because of the large number of patterns, 46 types, there is a mixture of static and dynamic, and the dynamic ones have been excluded in most studies. Few researchers have been working...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/3/826 |
| _version_ | 1827354549209268224 |
|---|---|
| author | Manato Kakizaki Abu Saleh Musa Miah Koki Hirooka Jungpil Shin |
| author_facet | Manato Kakizaki Abu Saleh Musa Miah Koki Hirooka Jungpil Shin |
| author_sort | Manato Kakizaki |
| collection | DOAJ |
| description | Japanese Sign Language (JSL) is vital for communication in Japan’s deaf and hard-of-hearing community. But probably because of the large number of patterns, 46 types, there is a mixture of static and dynamic, and the dynamic ones have been excluded in most studies. Few researchers have been working to develop a dynamic JSL alphabet, and their performance accuracy is unsatisfactory. We proposed a dynamic JSL recognition system using effective feature extraction and feature selection approaches to overcome the challenges. In the procedure, we follow the hand pose estimation, effective feature extraction, and machine learning techniques. We collected a video dataset capturing JSL gestures through standard RGB cameras and employed MediaPipe for hand pose estimation. Four types of features were proposed. The significance of these features is that the same feature generation method can be used regardless of the number of frames or whether the features are dynamic or static. We employed a Random forest (RF) based feature selection approach to select the potential feature. Finally, we fed the reduced features into the kernels-based Support Vector Machine (SVM) algorithm classification. Evaluations conducted on our proprietary newly created dynamic Japanese sign language alphabet dataset and LSA64 dynamic dataset yielded recognition accuracies of 97.20% and 98.40%, respectively. This innovative approach not only addresses the complexities of JSL but also holds the potential to bridge communication gaps, offering effective communication for the deaf and hard-of-hearing, and has broader implications for sign language recognition systems globally. |
| first_indexed | 2024-03-08T03:49:14Z |
| format | Article |
| id | doaj.art-d68b8f17746e49b086ff29133d0b1c38 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-08T03:49:14Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-d68b8f17746e49b086ff29133d0b1c382024-02-09T15:21:57ZengMDPI AGSensors1424-82202024-01-0124382610.3390/s24030826Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification ApproachManato Kakizaki0Abu Saleh Musa Miah1Koki Hirooka2Jungpil Shin3School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu 965-8580, JapanSchool of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu 965-8580, JapanSchool of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu 965-8580, JapanSchool of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu 965-8580, JapanJapanese Sign Language (JSL) is vital for communication in Japan’s deaf and hard-of-hearing community. But probably because of the large number of patterns, 46 types, there is a mixture of static and dynamic, and the dynamic ones have been excluded in most studies. Few researchers have been working to develop a dynamic JSL alphabet, and their performance accuracy is unsatisfactory. We proposed a dynamic JSL recognition system using effective feature extraction and feature selection approaches to overcome the challenges. In the procedure, we follow the hand pose estimation, effective feature extraction, and machine learning techniques. We collected a video dataset capturing JSL gestures through standard RGB cameras and employed MediaPipe for hand pose estimation. Four types of features were proposed. The significance of these features is that the same feature generation method can be used regardless of the number of frames or whether the features are dynamic or static. We employed a Random forest (RF) based feature selection approach to select the potential feature. Finally, we fed the reduced features into the kernels-based Support Vector Machine (SVM) algorithm classification. Evaluations conducted on our proprietary newly created dynamic Japanese sign language alphabet dataset and LSA64 dynamic dataset yielded recognition accuracies of 97.20% and 98.40%, respectively. This innovative approach not only addresses the complexities of JSL but also holds the potential to bridge communication gaps, offering effective communication for the deaf and hard-of-hearing, and has broader implications for sign language recognition systems globally.https://www.mdpi.com/1424-8220/24/3/826dynamic hand gesture recognitionjapanese sign language (JSL)effective feature selectionmachine learninghand skeleton points |
| spellingShingle | Manato Kakizaki Abu Saleh Musa Miah Koki Hirooka Jungpil Shin Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach Sensors dynamic hand gesture recognition japanese sign language (JSL) effective feature selection machine learning hand skeleton points |
| title | Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach |
| title_full | Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach |
| title_fullStr | Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach |
| title_full_unstemmed | Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach |
| title_short | Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach |
| title_sort | dynamic japanese sign language recognition throw hand pose estimation using effective feature extraction and classification approach |
| topic | dynamic hand gesture recognition japanese sign language (JSL) effective feature selection machine learning hand skeleton points |
| url | https://www.mdpi.com/1424-8220/24/3/826 |
| work_keys_str_mv | AT manatokakizaki dynamicjapanesesignlanguagerecognitionthrowhandposeestimationusingeffectivefeatureextractionandclassificationapproach AT abusalehmusamiah dynamicjapanesesignlanguagerecognitionthrowhandposeestimationusingeffectivefeatureextractionandclassificationapproach AT kokihirooka dynamicjapanesesignlanguagerecognitionthrowhandposeestimationusingeffectivefeatureextractionandclassificationapproach AT jungpilshin dynamicjapanesesignlanguagerecognitionthrowhandposeestimationusingeffectivefeatureextractionandclassificationapproach |