Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques
Air writing is one of the essential fields that the world is turning to, which can benefit from the world of the metaverse, as well as the ease of communication between humans and machines. The research literature on air writing and its applications shows significant work in English and Chinese, whi...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/23/9475 |
| _version_ | 1797399572709900288 |
|---|---|
| author | Khalid M. O. Nahar Izzat Alsmadi Rabia Emhamed Al Mamlook Ahmad Nasayreh Hasan Gharaibeh Ali Saeed Almuflih Fahad Alasim |
| author_facet | Khalid M. O. Nahar Izzat Alsmadi Rabia Emhamed Al Mamlook Ahmad Nasayreh Hasan Gharaibeh Ali Saeed Almuflih Fahad Alasim |
| author_sort | Khalid M. O. Nahar |
| collection | DOAJ |
| description | Air writing is one of the essential fields that the world is turning to, which can benefit from the world of the metaverse, as well as the ease of communication between humans and machines. The research literature on air writing and its applications shows significant work in English and Chinese, while little research is conducted in other languages, such as Arabic. To fill this gap, we propose a hybrid model that combines feature extraction with deep learning models and then uses machine learning (ML) and optical character recognition (OCR) methods and applies grid and random search optimization algorithms to obtain the best model parameters and outcomes. Several machine learning methods (e.g., neural networks (NNs), random forest (RF), K-nearest neighbours (KNN), and support vector machine (SVM)) are applied to deep features extracted from deep convolutional neural networks (CNNs), such as VGG16, VGG19, and SqueezeNet. Our study uses the AHAWP dataset, which consists of diverse writing styles and hand sign variations, to train and evaluate the models. Prepossessing schemes are applied to improve data quality by reducing bias. Furthermore, OCR character (OCR) methods are integrated into our model to isolate individual letters from continuous air-written gestures and improve recognition results. The results of this study showed that the proposed model achieved the best accuracy of 88.8% using NN with VGG16. |
| first_indexed | 2024-03-09T01:43:07Z |
| format | Article |
| id | doaj.art-7fcf98c5bd234ac7b8a77dbf2322b550 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T01:43:07Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-7fcf98c5bd234ac7b8a77dbf2322b5502023-12-08T15:26:08ZengMDPI AGSensors1424-82202023-11-012323947510.3390/s23239475Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) TechniquesKhalid M. O. Nahar0Izzat Alsmadi1Rabia Emhamed Al Mamlook2Ahmad Nasayreh3Hasan Gharaibeh4Ali Saeed Almuflih5Fahad Alasim6Computer Science Department, Faculty of Information Technology and Computer Sciences, Yarmouk University, Irbid 21163, JordanDepartment of Computing and Cyber Security, Texas A&M University-San Antonio, San Antonio, TX 78224, USADepartment of Business Administration, Trine University, Angola, IN 49008, USAComputer Science Department, Faculty of Information Technology and Computer Sciences, Yarmouk University, Irbid 21163, JordanComputer Science Department, Faculty of Information Technology and Computer Sciences, Yarmouk University, Irbid 21163, JordanDepartment of Industrial Engineering, College of Engineering, King Khalid University, Abha 62529, Saudi ArabiaDepartment of Industrial Engineering, College of Engineering, King Saud University, Riyadh 11495, Saudi ArabiaAir writing is one of the essential fields that the world is turning to, which can benefit from the world of the metaverse, as well as the ease of communication between humans and machines. The research literature on air writing and its applications shows significant work in English and Chinese, while little research is conducted in other languages, such as Arabic. To fill this gap, we propose a hybrid model that combines feature extraction with deep learning models and then uses machine learning (ML) and optical character recognition (OCR) methods and applies grid and random search optimization algorithms to obtain the best model parameters and outcomes. Several machine learning methods (e.g., neural networks (NNs), random forest (RF), K-nearest neighbours (KNN), and support vector machine (SVM)) are applied to deep features extracted from deep convolutional neural networks (CNNs), such as VGG16, VGG19, and SqueezeNet. Our study uses the AHAWP dataset, which consists of diverse writing styles and hand sign variations, to train and evaluate the models. Prepossessing schemes are applied to improve data quality by reducing bias. Furthermore, OCR character (OCR) methods are integrated into our model to isolate individual letters from continuous air-written gestures and improve recognition results. The results of this study showed that the proposed model achieved the best accuracy of 88.8% using NN with VGG16.https://www.mdpi.com/1424-8220/23/23/9475Arabic air-writing recognitionmachine learningOCRrecognitiondeep learning |
| spellingShingle | Khalid M. O. Nahar Izzat Alsmadi Rabia Emhamed Al Mamlook Ahmad Nasayreh Hasan Gharaibeh Ali Saeed Almuflih Fahad Alasim Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques Sensors Arabic air-writing recognition machine learning OCR recognition deep learning |
| title | Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques |
| title_full | Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques |
| title_fullStr | Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques |
| title_full_unstemmed | Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques |
| title_short | Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques |
| title_sort | recognition of arabic air written letters machine learning convolutional neural networks and optical character recognition ocr techniques |
| topic | Arabic air-writing recognition machine learning OCR recognition deep learning |
| url | https://www.mdpi.com/1424-8220/23/23/9475 |
| work_keys_str_mv | AT khalidmonahar recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques AT izzatalsmadi recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques AT rabiaemhamedalmamlook recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques AT ahmadnasayreh recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques AT hasangharaibeh recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques AT alisaeedalmuflih recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques AT fahadalasim recognitionofarabicairwrittenlettersmachinelearningconvolutionalneuralnetworksandopticalcharacterrecognitionocrtechniques |