Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network
Hand gesture recognition is one of the most effective modes of interaction between humans and computers due to being highly flexible and user-friendly. A real-time hand gesture recognition system should aim to develop a user-independent interface with high recognition performance. Nowadays, convolut...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/3/706 |
| _version_ | 1827658911989104640 |
|---|---|
| author | Jaya Prakash Sahoo Allam Jaya Prakash Paweł Pławiak Saunak Samantray |
| author_facet | Jaya Prakash Sahoo Allam Jaya Prakash Paweł Pławiak Saunak Samantray |
| author_sort | Jaya Prakash Sahoo |
| collection | DOAJ |
| description | Hand gesture recognition is one of the most effective modes of interaction between humans and computers due to being highly flexible and user-friendly. A real-time hand gesture recognition system should aim to develop a user-independent interface with high recognition performance. Nowadays, convolutional neural networks (CNNs) show high recognition rates in image classification problems. Due to the unavailability of large labeled image samples in static hand gesture images, it is a challenging task to train deep CNN networks such as AlexNet, VGG-16 and ResNet from scratch. Therefore, inspired by CNN performance, an end-to-end fine-tuning method of a pre-trained CNN model with score-level fusion technique is proposed here to recognize hand gestures in a dataset with a low number of gesture images. The effectiveness of the proposed technique is evaluated using leave-one-subject-out cross-validation (LOO CV) and regular CV tests on two benchmark datasets. A real-time American sign language (ASL) recognition system is developed and tested using the proposed technique. |
| first_indexed | 2024-03-09T23:12:15Z |
| format | Article |
| id | doaj.art-e93df47cf8fa46018f5fe4dd2f06f05b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T23:12:15Z |
| publishDate | 2022-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-e93df47cf8fa46018f5fe4dd2f06f05b2023-11-23T17:43:57ZengMDPI AGSensors1424-82202022-01-0122370610.3390/s22030706Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural NetworkJaya Prakash Sahoo0Allam Jaya Prakash1Paweł Pławiak2Saunak Samantray3Department of Electronics and Communication Engineering, National Institute of Technology, Rourkela 769008, Odisha, IndiaDepartment of Electronics and Communication Engineering, National Institute of Technology, Rourkela 769008, Odisha, IndiaDepartment of Computer Science, Faculty of Computer Science and Telecommunications, Cracow University of Technology, Warszawska 24, 31-155 Krakow, PolandDepartment of Electronics and Tele Communication Engineering, IIIT Bhubaneswar, Bhubaneswar 751003, Odisha, IndiaHand gesture recognition is one of the most effective modes of interaction between humans and computers due to being highly flexible and user-friendly. A real-time hand gesture recognition system should aim to develop a user-independent interface with high recognition performance. Nowadays, convolutional neural networks (CNNs) show high recognition rates in image classification problems. Due to the unavailability of large labeled image samples in static hand gesture images, it is a challenging task to train deep CNN networks such as AlexNet, VGG-16 and ResNet from scratch. Therefore, inspired by CNN performance, an end-to-end fine-tuning method of a pre-trained CNN model with score-level fusion technique is proposed here to recognize hand gestures in a dataset with a low number of gesture images. The effectiveness of the proposed technique is evaluated using leave-one-subject-out cross-validation (LOO CV) and regular CV tests on two benchmark datasets. A real-time American sign language (ASL) recognition system is developed and tested using the proposed technique.https://www.mdpi.com/1424-8220/22/3/706ASLfine-tunninghand gesture recognitionpre-trained CNNreal-time gesture recognitionscore fusion |
| spellingShingle | Jaya Prakash Sahoo Allam Jaya Prakash Paweł Pławiak Saunak Samantray Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network Sensors ASL fine-tunning hand gesture recognition pre-trained CNN real-time gesture recognition score fusion |
| title | Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network |
| title_full | Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network |
| title_fullStr | Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network |
| title_full_unstemmed | Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network |
| title_short | Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network |
| title_sort | real time hand gesture recognition using fine tuned convolutional neural network |
| topic | ASL fine-tunning hand gesture recognition pre-trained CNN real-time gesture recognition score fusion |
| url | https://www.mdpi.com/1424-8220/22/3/706 |
| work_keys_str_mv | AT jayaprakashsahoo realtimehandgesturerecognitionusingfinetunedconvolutionalneuralnetwork AT allamjayaprakash realtimehandgesturerecognitionusingfinetunedconvolutionalneuralnetwork AT pawełpławiak realtimehandgesturerecognitionusingfinetunedconvolutionalneuralnetwork AT saunaksamantray realtimehandgesturerecognitionusingfinetunedconvolutionalneuralnetwork |