Indian Sign Language recognition system using SURF with SVM and CNN
Hand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to thi...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-07-01
|
| Series: | Array |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2590005622000121 |
| _version_ | 1828199771756560384 |
|---|---|
| author | Shagun Katoch Varsha Singh Uma Shanker Tiwary |
| author_facet | Shagun Katoch Varsha Singh Uma Shanker Tiwary |
| author_sort | Shagun Katoch |
| collection | DOAJ |
| description | Hand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to this category. This is the key reason why it would have a huge beneficial effect on these individuals to incorporate a framework that would understand Indian Sign Language. In this paper, we present a technique that uses the Bag of Visual Words model (BOVW) to recognize Indian sign language alphabets (A-Z) and digits (0–9) in a live video stream and output the predicted labels in the form of text as well as speech. Segmentation is done based on skin colour as well as background subtraction. SURF (Speeded Up Robust Features) features have been extracted from the images and histograms are generated to map the signs with corresponding labels. The Support Vector Machine (SVM) and Convolutional Neural Networks (CNN) are used for classification. An interactive Graphical User Interface (GUI) is also developed for easy access. |
| first_indexed | 2024-04-12T11:02:14Z |
| format | Article |
| id | doaj.art-379c160eaeee403abbcb665edf504917 |
| institution | Directory Open Access Journal |
| issn | 2590-0056 |
| language | English |
| last_indexed | 2024-04-12T11:02:14Z |
| publishDate | 2022-07-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Array |
| spelling | doaj.art-379c160eaeee403abbcb665edf5049172022-12-22T03:35:54ZengElsevierArray2590-00562022-07-0114100141Indian Sign Language recognition system using SURF with SVM and CNNShagun Katoch0Varsha Singh1Uma Shanker Tiwary2Computer Science, National Institute of Technology, Hamirpur, IndiaDepartment of Information Technology, Indian Institute of Information Technology, Allahabad, India; Corresponding author.Department of Information Technology, Indian Institute of Information Technology, Allahabad, IndiaHand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to this category. This is the key reason why it would have a huge beneficial effect on these individuals to incorporate a framework that would understand Indian Sign Language. In this paper, we present a technique that uses the Bag of Visual Words model (BOVW) to recognize Indian sign language alphabets (A-Z) and digits (0–9) in a live video stream and output the predicted labels in the form of text as well as speech. Segmentation is done based on skin colour as well as background subtraction. SURF (Speeded Up Robust Features) features have been extracted from the images and histograms are generated to map the signs with corresponding labels. The Support Vector Machine (SVM) and Convolutional Neural Networks (CNN) are used for classification. An interactive Graphical User Interface (GUI) is also developed for easy access.http://www.sciencedirect.com/science/article/pii/S2590005622000121Hand sign recognitionIndian sign language (ISL)Bag of visual words (BOVW)SURF featuresSVMCNN |
| spellingShingle | Shagun Katoch Varsha Singh Uma Shanker Tiwary Indian Sign Language recognition system using SURF with SVM and CNN Array Hand sign recognition Indian sign language (ISL) Bag of visual words (BOVW) SURF features SVM CNN |
| title | Indian Sign Language recognition system using SURF with SVM and CNN |
| title_full | Indian Sign Language recognition system using SURF with SVM and CNN |
| title_fullStr | Indian Sign Language recognition system using SURF with SVM and CNN |
| title_full_unstemmed | Indian Sign Language recognition system using SURF with SVM and CNN |
| title_short | Indian Sign Language recognition system using SURF with SVM and CNN |
| title_sort | indian sign language recognition system using surf with svm and cnn |
| topic | Hand sign recognition Indian sign language (ISL) Bag of visual words (BOVW) SURF features SVM CNN |
| url | http://www.sciencedirect.com/science/article/pii/S2590005622000121 |
| work_keys_str_mv | AT shagunkatoch indiansignlanguagerecognitionsystemusingsurfwithsvmandcnn AT varshasingh indiansignlanguagerecognitionsystemusingsurfwithsvmandcnn AT umashankertiwary indiansignlanguagerecognitionsystemusingsurfwithsvmandcnn |