Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features
With the latest advancements, hand gesture recognition is becoming an effective way of communication and gaining popularity from a research point of view. Hearing impaired people around the world need assistance, while sign language is only understood by a few people around the globe. It becomes cha...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10198393/ |
| _version_ | 1797742948834607104 |
|---|---|
| author | Hira Ansar Naif Al Mudawi Saud S. Alotaibi Abdulwahab Alazeb Bayan Ibrahimm Alabdullah Mohammed Alonazi Jeongmin Park |
| author_facet | Hira Ansar Naif Al Mudawi Saud S. Alotaibi Abdulwahab Alazeb Bayan Ibrahimm Alabdullah Mohammed Alonazi Jeongmin Park |
| author_sort | Hira Ansar |
| collection | DOAJ |
| description | With the latest advancements, hand gesture recognition is becoming an effective way of communication and gaining popularity from a research point of view. Hearing impaired people around the world need assistance, while sign language is only understood by a few people around the globe. It becomes challenging for untrained people to communicate easily, research community has tried to train systems with a variety of models to facilitate communication with hearing impaired people and also human-computer interaction. Researchers have detected gestures with numerous recognition rates; however, the recognition rate still needs improvement. As the images captured via cameras possess multiple issues, the light intensity variation makes it a challenging task to extract gestures from such images, extra information in captured images, such as noise hinders the computation time, and complex backgrounds make the extraction of gestures difficult. A novel approach is proposed in this paper for character detection and recognition. The proposed system is divided into five steps for hand gesture recognition. Firstly, images are pre-processed to reduce noise and intensity is adjusted. The pre-processed images region of interest is detected via directional images. After hand extraction, landmarks are extracted via a convex hull. Each gesture is used to extract geometric features for the proposed hand gesture recognition (HGR) system. The extracted features helped in gesture detection and recognition via the Convolutional Neural Network (CNN) classifier. The proposed approach experimentation result demonstrated over the MNIST dataset achieved a gesture recognition rate of 93.2% and 90.2% with one-third and two-third training validation systems, respectively. Also, the proposed system performance is validated on the ASL dataset, giving accuracy of 91.6% and 88.14% with one-third and two-third training validation systems, respectively. The proposed system is also compared with other conventional systems. Different emerging domains such as human-computer interaction (HCI), human-robot interaction (HRI), and virtual reality (VR) are applicable to the proposed system to fill the communication gap. |
| first_indexed | 2024-03-12T14:47:32Z |
| format | Article |
| id | doaj.art-1ba77d09823f430d8b813ac6a72cc5ae |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T14:47:32Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1ba77d09823f430d8b813ac6a72cc5ae2023-08-15T23:01:30ZengIEEEIEEE Access2169-35362023-01-0111820658207810.1109/ACCESS.2023.330071210198393Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric FeaturesHira Ansar0Naif Al Mudawi1Saud S. Alotaibi2Abdulwahab Alazeb3Bayan Ibrahimm Alabdullah4Mohammed Alonazi5Jeongmin Park6https://orcid.org/0000-0001-8027-0876Faculty of Computer Science and AI, Air University, Islamabad, PakistanDepartment of Computer Science, College of Computer Science and Information System, Najran University, Najran, Saudi ArabiaInformation Systems Department, Umm Al-Qura University, Mecca, Saudi ArabiaDepartment of Computer Science, College of Computer Science and Information System, Najran University, Najran, Saudi ArabiaDepartment of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh, Saudi ArabiaDepartment of Information Systems, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-Kharj, Saudi ArabiaDepartment of Computer Engineering, Tech University of Korea, Siheung-si, South KoreaWith the latest advancements, hand gesture recognition is becoming an effective way of communication and gaining popularity from a research point of view. Hearing impaired people around the world need assistance, while sign language is only understood by a few people around the globe. It becomes challenging for untrained people to communicate easily, research community has tried to train systems with a variety of models to facilitate communication with hearing impaired people and also human-computer interaction. Researchers have detected gestures with numerous recognition rates; however, the recognition rate still needs improvement. As the images captured via cameras possess multiple issues, the light intensity variation makes it a challenging task to extract gestures from such images, extra information in captured images, such as noise hinders the computation time, and complex backgrounds make the extraction of gestures difficult. A novel approach is proposed in this paper for character detection and recognition. The proposed system is divided into five steps for hand gesture recognition. Firstly, images are pre-processed to reduce noise and intensity is adjusted. The pre-processed images region of interest is detected via directional images. After hand extraction, landmarks are extracted via a convex hull. Each gesture is used to extract geometric features for the proposed hand gesture recognition (HGR) system. The extracted features helped in gesture detection and recognition via the Convolutional Neural Network (CNN) classifier. The proposed approach experimentation result demonstrated over the MNIST dataset achieved a gesture recognition rate of 93.2% and 90.2% with one-third and two-third training validation systems, respectively. Also, the proposed system performance is validated on the ASL dataset, giving accuracy of 91.6% and 88.14% with one-third and two-third training validation systems, respectively. The proposed system is also compared with other conventional systems. Different emerging domains such as human-computer interaction (HCI), human-robot interaction (HRI), and virtual reality (VR) are applicable to the proposed system to fill the communication gap.https://ieeexplore.ieee.org/document/10198393/ASL sign languagecharacter understandinglandmark identificationgeometric featurehand gesture recognitionCNN |
| spellingShingle | Hira Ansar Naif Al Mudawi Saud S. Alotaibi Abdulwahab Alazeb Bayan Ibrahimm Alabdullah Mohammed Alonazi Jeongmin Park Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features IEEE Access ASL sign language character understanding landmark identification geometric feature hand gesture recognition CNN |
| title | Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features |
| title_full | Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features |
| title_fullStr | Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features |
| title_full_unstemmed | Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features |
| title_short | Hand Gesture Recognition for Characters Understanding Using Convex Hull Landmarks and Geometric Features |
| title_sort | hand gesture recognition for characters understanding using convex hull landmarks and geometric features |
| topic | ASL sign language character understanding landmark identification geometric feature hand gesture recognition CNN |
| url | https://ieeexplore.ieee.org/document/10198393/ |
| work_keys_str_mv | AT hiraansar handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures AT naifalmudawi handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures AT saudsalotaibi handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures AT abdulwahabalazeb handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures AT bayanibrahimmalabdullah handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures AT mohammedalonazi handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures AT jeongminpark handgesturerecognitionforcharactersunderstandingusingconvexhulllandmarksandgeometricfeatures |