SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition
Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestur...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/22/12204 |
| _version_ | 1797460302675050496 |
|---|---|
| author | Chun Keat Tan Kian Ming Lim Chin Poo Lee Roy Kwang Yang Chang Ali Alqahtani |
| author_facet | Chun Keat Tan Kian Ming Lim Chin Poo Lee Roy Kwang Yang Chang Ali Alqahtani |
| author_sort | Chun Keat Tan |
| collection | DOAJ |
| description | Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively. |
| first_indexed | 2024-03-09T17:03:17Z |
| format | Article |
| id | doaj.art-2e7cd8e0c38d4377ac5bef0975d5b39c |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T17:03:17Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-2e7cd8e0c38d4377ac5bef0975d5b39c2023-11-24T14:26:40ZengMDPI AGApplied Sciences2076-34172023-11-0113221220410.3390/app132212204SDViT: Stacking of Distilled Vision Transformers for Hand Gesture RecognitionChun Keat Tan0Kian Ming Lim1Chin Poo Lee2Roy Kwang Yang Chang3Ali Alqahtani4Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Malacca 75450, MalaysiaFaculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Malacca 75450, MalaysiaFaculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Malacca 75450, MalaysiaFaculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Malacca 75450, MalaysiaDepartment of Computer Science, King Khalid University, Abha 61421, Saudi ArabiaHand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively.https://www.mdpi.com/2076-3417/13/22/12204hand gesture recognitionsign language recognitionvision transformerknowledge distillationstacking |
| spellingShingle | Chun Keat Tan Kian Ming Lim Chin Poo Lee Roy Kwang Yang Chang Ali Alqahtani SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition Applied Sciences hand gesture recognition sign language recognition vision transformer knowledge distillation stacking |
| title | SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition |
| title_full | SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition |
| title_fullStr | SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition |
| title_full_unstemmed | SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition |
| title_short | SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition |
| title_sort | sdvit stacking of distilled vision transformers for hand gesture recognition |
| topic | hand gesture recognition sign language recognition vision transformer knowledge distillation stacking |
| url | https://www.mdpi.com/2076-3417/13/22/12204 |
| work_keys_str_mv | AT chunkeattan sdvitstackingofdistilledvisiontransformersforhandgesturerecognition AT kianminglim sdvitstackingofdistilledvisiontransformersforhandgesturerecognition AT chinpoolee sdvitstackingofdistilledvisiontransformersforhandgesturerecognition AT roykwangyangchang sdvitstackingofdistilledvisiontransformersforhandgesturerecognition AT alialqahtani sdvitstackingofdistilledvisiontransformersforhandgesturerecognition |