Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model
Using gestures can help people with certain disabilities in communicating with other people. This paper proposes a lightweight model based on YOLO (You Only Look Once) v3 and DarkNet-53 convolutional neural networks for gesture recognition without additional preprocessing, image filtering, and enhan...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-05-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/9/4164 |
| _version_ | 1797535433247162368 |
|---|---|
| author | Abdullah Mujahid Mazhar Javed Awan Awais Yasin Mazin Abed Mohammed Robertas Damaševičius Rytis Maskeliūnas Karrar Hameed Abdulkareem |
| author_facet | Abdullah Mujahid Mazhar Javed Awan Awais Yasin Mazin Abed Mohammed Robertas Damaševičius Rytis Maskeliūnas Karrar Hameed Abdulkareem |
| author_sort | Abdullah Mujahid |
| collection | DOAJ |
| description | Using gestures can help people with certain disabilities in communicating with other people. This paper proposes a lightweight model based on YOLO (You Only Look Once) v3 and DarkNet-53 convolutional neural networks for gesture recognition without additional preprocessing, image filtering, and enhancement of images. The proposed model achieved high accuracy even in a complex environment, and it successfully detected gestures even in low-resolution picture mode. The proposed model was evaluated on a labeled dataset of hand gestures in both Pascal VOC and YOLO format. We achieved better results by extracting features from the hand and recognized hand gestures of our proposed YOLOv3 based model with accuracy, precision, recall, and an F-1 score of 97.68, 94.88, 98.66, and 96.70%, respectively. Further, we compared our model with Single Shot Detector (SSD) and Visual Geometry Group (VGG16), which achieved an accuracy between 82 and 85%. The trained model can be used for real-time detection, both for static hand images and dynamic gestures recorded on a video. |
| first_indexed | 2024-03-10T11:44:26Z |
| format | Article |
| id | doaj.art-8a67347cb7d54be8b7a796e03b5b585e |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T11:44:26Z |
| publishDate | 2021-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-8a67347cb7d54be8b7a796e03b5b585e2023-11-21T18:11:59ZengMDPI AGApplied Sciences2076-34172021-05-01119416410.3390/app11094164Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 ModelAbdullah Mujahid0Mazhar Javed Awan1Awais Yasin2Mazin Abed Mohammed3Robertas Damaševičius4Rytis Maskeliūnas5Karrar Hameed Abdulkareem6Department of Computer Science, University of Management and Technology, Lahore 54770, PakistanDepartment of Software Engineering, University of Management and Technology, Lahore 54770, PakistanDepartment of Computer Engineering, National University of Technology, Islamabad 44000, PakistanInformation Systems Department, College of Computer Science and Information Technology, University of Anbar, Anbar 31001, IraqFaculty of Applied Mathematics, Silesian University of Technology, 44-100 Gliwice, PolandDepartment of Applied Informatics, Vytautas Magnus University, 44404 Kaunas, LithuaniaCollege of Agriculture, Al-Muthanna University, Samawah 66001, IraqUsing gestures can help people with certain disabilities in communicating with other people. This paper proposes a lightweight model based on YOLO (You Only Look Once) v3 and DarkNet-53 convolutional neural networks for gesture recognition without additional preprocessing, image filtering, and enhancement of images. The proposed model achieved high accuracy even in a complex environment, and it successfully detected gestures even in low-resolution picture mode. The proposed model was evaluated on a labeled dataset of hand gestures in both Pascal VOC and YOLO format. We achieved better results by extracting features from the hand and recognized hand gestures of our proposed YOLOv3 based model with accuracy, precision, recall, and an F-1 score of 97.68, 94.88, 98.66, and 96.70%, respectively. Further, we compared our model with Single Shot Detector (SSD) and Visual Geometry Group (VGG16), which achieved an accuracy between 82 and 85%. The trained model can be used for real-time detection, both for static hand images and dynamic gestures recorded on a video.https://www.mdpi.com/2076-3417/11/9/4164convolutional neural networkhand gesturedigital image processingYOLOv3artificial intelligence |
| spellingShingle | Abdullah Mujahid Mazhar Javed Awan Awais Yasin Mazin Abed Mohammed Robertas Damaševičius Rytis Maskeliūnas Karrar Hameed Abdulkareem Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model Applied Sciences convolutional neural network hand gesture digital image processing YOLOv3 artificial intelligence |
| title | Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model |
| title_full | Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model |
| title_fullStr | Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model |
| title_full_unstemmed | Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model |
| title_short | Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model |
| title_sort | real time hand gesture recognition based on deep learning yolov3 model |
| topic | convolutional neural network hand gesture digital image processing YOLOv3 artificial intelligence |
| url | https://www.mdpi.com/2076-3417/11/9/4164 |
| work_keys_str_mv | AT abdullahmujahid realtimehandgesturerecognitionbasedondeeplearningyolov3model AT mazharjavedawan realtimehandgesturerecognitionbasedondeeplearningyolov3model AT awaisyasin realtimehandgesturerecognitionbasedondeeplearningyolov3model AT mazinabedmohammed realtimehandgesturerecognitionbasedondeeplearningyolov3model AT robertasdamasevicius realtimehandgesturerecognitionbasedondeeplearningyolov3model AT rytismaskeliunas realtimehandgesturerecognitionbasedondeeplearningyolov3model AT karrarhameedabdulkareem realtimehandgesturerecognitionbasedondeeplearningyolov3model |