Facilitating the communication with deaf people: Building a largest Saudi sign language dataset
Recently, several countries have been trying hard to facilitate the integration of disabled people into their societies by ensuring equal opportunities through ease of access to social services, daily human necessities, and the labor market. Deafness is considered one of the major disabilities separ...
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-09-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157823001969 |
| _version_ | 1797663883007098880 |
|---|---|
| author | Mansour Alsulaiman Mohammed Faisal Mohamed Mekhtiche Mohamed Bencherif Tariq Alrayes Ghulam Muhammad Hassan Mathkour Wadood Abdul Yousef Alohali Mansour Alqahtani Habib Al-Habib Hassan Alhalafi Mohammed Algabri Muneer Al-hammadi Hamdi Altaheri Taha Alfakih |
| author_facet | Mansour Alsulaiman Mohammed Faisal Mohamed Mekhtiche Mohamed Bencherif Tariq Alrayes Ghulam Muhammad Hassan Mathkour Wadood Abdul Yousef Alohali Mansour Alqahtani Habib Al-Habib Hassan Alhalafi Mohammed Algabri Muneer Al-hammadi Hamdi Altaheri Taha Alfakih |
| author_sort | Mansour Alsulaiman |
| collection | DOAJ |
| description | Recently, several countries have been trying hard to facilitate the integration of disabled people into their societies by ensuring equal opportunities through ease of access to social services, daily human necessities, and the labor market. Deafness is considered one of the major disabilities separating the deaf from their society. To integrate the deaf fully into society, a two-way mode of communication is required: one from the deaf to the hearing people, and the other from the hearing to the deaf. Communication from the hearing person to the deaf is generally easy and can be done through speech recognition and text-to-sign representations, but communication from the deaf to the hearing is somewhat difficult and requires a sign recognition module that recognizes the sign motions from the deaf and translates it to a text; following this, a speech synthesis module will translate this text to speech. To build the sign recognition module, a sign language dataset is required. This paper contributes to the literature by introducing a comprehensive survey of 17 Arabic sign language datasets and by developing a well-organized framework that is used to build a sign language dataset. This paper also contributes to the literature by creating the largest Saudi Sign Language (SSL) database—the King Saud University Saudi-SSL (KSU-SSL data-base)—with 293 signs, 33 signers, 145,035 samples, and 10 domains (healthcare, common, alpha-bets, verbs, pronouns and adverbs, numbers, days, kings, family, and regions). This paper also contributes to the literature by introducing a convolutional graph neural network (CGCN) architecture for sign language recognition and applying the proposed architecture to the built KSU-SSL database. The architecture is made up of a small number of separable 3DGCN layers, and is augmented with a spatial attention mechanism. This study is a part of the project that aims to develop a two-way communication system for Saudi deaf individuals. |
| first_indexed | 2024-03-11T19:21:14Z |
| format | Article |
| id | doaj.art-894b32a084d541b08016120f241ad296 |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-03-11T19:21:14Z |
| publishDate | 2023-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-894b32a084d541b08016120f241ad2962023-10-07T04:33:56ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782023-09-01358101642Facilitating the communication with deaf people: Building a largest Saudi sign language datasetMansour Alsulaiman0Mohammed Faisal1Mohamed Mekhtiche2Mohamed Bencherif3Tariq Alrayes4Ghulam Muhammad5Hassan Mathkour6Wadood Abdul7Yousef Alohali8Mansour Alqahtani9Habib Al-Habib10Hassan Alhalafi11Mohammed Algabri12Muneer Al-hammadi13Hamdi Altaheri14Taha Alfakih15Center of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi Arabia; Corresponding authors at: Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi Arabia.Center of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Center of AI & Robotics, Kuwait College of Science and Technology (KCST), Kuwait; Corresponding authors at: Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi Arabia.Center of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi ArabiaDepartment of Special Education, College of Education, King Saud University, Riyadh, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Science Department, College of Computer and Information Science, King Saud University, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Science Department, College of Computer and Information Science, King Saud University, Saudi ArabiaHigher Education Program for Deaf and Hard-of-Hearing Students, King Saud University, Riyadh, Saudi ArabiaHigher Education Program for Deaf and Hard-of-Hearing Students, King Saud University, Riyadh, Saudi ArabiaHigher Education Program for Deaf and Hard-of-Hearing Students, King Saud University, Riyadh, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Science and Information Systems Department, College of Applied Sciences, AlMaarefa University, Riyadh 13713, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Department of Civil and Environmental Engineering, Faculty of Engineering, Norwegian University of Science and Technology, Høgskoleringen 1, 7034 Trondheim, NorwayCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi Arabia; Computer Engineering Department, College of Computer and Information Science, King Saud University, Riyadh 1543, Saudi ArabiaCenter of Smart Robotics Research, College of Computer and Information Science, King Saud University, Saudi ArabiaRecently, several countries have been trying hard to facilitate the integration of disabled people into their societies by ensuring equal opportunities through ease of access to social services, daily human necessities, and the labor market. Deafness is considered one of the major disabilities separating the deaf from their society. To integrate the deaf fully into society, a two-way mode of communication is required: one from the deaf to the hearing people, and the other from the hearing to the deaf. Communication from the hearing person to the deaf is generally easy and can be done through speech recognition and text-to-sign representations, but communication from the deaf to the hearing is somewhat difficult and requires a sign recognition module that recognizes the sign motions from the deaf and translates it to a text; following this, a speech synthesis module will translate this text to speech. To build the sign recognition module, a sign language dataset is required. This paper contributes to the literature by introducing a comprehensive survey of 17 Arabic sign language datasets and by developing a well-organized framework that is used to build a sign language dataset. This paper also contributes to the literature by creating the largest Saudi Sign Language (SSL) database—the King Saud University Saudi-SSL (KSU-SSL data-base)—with 293 signs, 33 signers, 145,035 samples, and 10 domains (healthcare, common, alpha-bets, verbs, pronouns and adverbs, numbers, days, kings, family, and regions). This paper also contributes to the literature by introducing a convolutional graph neural network (CGCN) architecture for sign language recognition and applying the proposed architecture to the built KSU-SSL database. The architecture is made up of a small number of separable 3DGCN layers, and is augmented with a spatial attention mechanism. This study is a part of the project that aims to develop a two-way communication system for Saudi deaf individuals.http://www.sciencedirect.com/science/article/pii/S1319157823001969Saudi DialectSaudi sign languageSign language datasetArabic sign language |
| spellingShingle | Mansour Alsulaiman Mohammed Faisal Mohamed Mekhtiche Mohamed Bencherif Tariq Alrayes Ghulam Muhammad Hassan Mathkour Wadood Abdul Yousef Alohali Mansour Alqahtani Habib Al-Habib Hassan Alhalafi Mohammed Algabri Muneer Al-hammadi Hamdi Altaheri Taha Alfakih Facilitating the communication with deaf people: Building a largest Saudi sign language dataset Journal of King Saud University: Computer and Information Sciences Saudi Dialect Saudi sign language Sign language dataset Arabic sign language |
| title | Facilitating the communication with deaf people: Building a largest Saudi sign language dataset |
| title_full | Facilitating the communication with deaf people: Building a largest Saudi sign language dataset |
| title_fullStr | Facilitating the communication with deaf people: Building a largest Saudi sign language dataset |
| title_full_unstemmed | Facilitating the communication with deaf people: Building a largest Saudi sign language dataset |
| title_short | Facilitating the communication with deaf people: Building a largest Saudi sign language dataset |
| title_sort | facilitating the communication with deaf people building a largest saudi sign language dataset |
| topic | Saudi Dialect Saudi sign language Sign language dataset Arabic sign language |
| url | http://www.sciencedirect.com/science/article/pii/S1319157823001969 |
| work_keys_str_mv | AT mansouralsulaiman facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT mohammedfaisal facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT mohamedmekhtiche facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT mohamedbencherif facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT tariqalrayes facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT ghulammuhammad facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT hassanmathkour facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT wadoodabdul facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT yousefalohali facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT mansouralqahtani facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT habibalhabib facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT hassanalhalafi facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT mohammedalgabri facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT muneeralhammadi facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT hamdialtaheri facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset AT tahaalfakih facilitatingthecommunicationwithdeafpeoplebuildingalargestsaudisignlanguagedataset |