A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician
Few hearing people know and use Mexican Sign Language (MSL). Consequently, this is the main barrier between people who having total or partial hearing loss and hearing people. This study proposes a system that recognizes and animates in real time a set of signs belonging to the semantic field of gen...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9970338/ |
| _version_ | 1811201179504345088 |
|---|---|
| author | Candy Obdulia Sosa-Jimenez Homero Vladimir Rios-Figueroa Ana Luisa Solis-Gonzalez-Cosio |
| author_facet | Candy Obdulia Sosa-Jimenez Homero Vladimir Rios-Figueroa Ana Luisa Solis-Gonzalez-Cosio |
| author_sort | Candy Obdulia Sosa-Jimenez |
| collection | DOAJ |
| description | Few hearing people know and use Mexican Sign Language (MSL). Consequently, this is the main barrier between people who having total or partial hearing loss and hearing people. This study proposes a system that recognizes and animates in real time a set of signs belonging to the semantic field of general medicine consultation services. Therefore, a linkage between a hearing doctor and a deaf patient can be established in a non-intrusive way and with easy dynamic interaction. Our main contribution is a bidirectional translator system for Mexican Sign Language in the context of primary care health services, in addition to basic signs to fingerspell alphabet and numbers as a complement to provide personal information such as name, age, etc. The recognition module uses a Microsoft Kinect sensor to obtain sign trajectories and images to feed hidden Markov Models (HMMs) for processing sign samples in real time. The experiments showed the recognition of 82 different signs by 22 participants. As a result, accuracy and F1 scores average rates of 99% and 88%, respectively, were obtained. |
| first_indexed | 2024-04-12T02:16:32Z |
| format | Article |
| id | doaj.art-345841f2292e43259314fe550b377734 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T02:16:32Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-345841f2292e43259314fe550b3777342022-12-22T03:52:14ZengIEEEIEEE Access2169-35362022-01-011012762012763510.1109/ACCESS.2022.32266969970338A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care PhysicianCandy Obdulia Sosa-Jimenez0https://orcid.org/0000-0002-5267-4643Homero Vladimir Rios-Figueroa1https://orcid.org/0000-0001-8673-8863Ana Luisa Solis-Gonzalez-Cosio2https://orcid.org/0000-0002-0266-3874School of Statistics and Informatics, University of Veracruz (UV), Xalapa, Veracruz, MexicoResearch Institute in Artificial Intelligence, University of Veracruz (UV), Xalapa, Veracruz, MexicoSchool of Sciences, National Autonomous University of Mexico (UNAM), Mexico City, MexicoFew hearing people know and use Mexican Sign Language (MSL). Consequently, this is the main barrier between people who having total or partial hearing loss and hearing people. This study proposes a system that recognizes and animates in real time a set of signs belonging to the semantic field of general medicine consultation services. Therefore, a linkage between a hearing doctor and a deaf patient can be established in a non-intrusive way and with easy dynamic interaction. Our main contribution is a bidirectional translator system for Mexican Sign Language in the context of primary care health services, in addition to basic signs to fingerspell alphabet and numbers as a complement to provide personal information such as name, age, etc. The recognition module uses a Microsoft Kinect sensor to obtain sign trajectories and images to feed hidden Markov Models (HMMs) for processing sign samples in real time. The experiments showed the recognition of 82 different signs by 22 participants. As a result, accuracy and F1 scores average rates of 99% and 88%, respectively, were obtained.https://ieeexplore.ieee.org/document/9970338/Mexican sign languagedepth sensordynamic sign language recognitionmedicine consultation servicessign language synthesis |
| spellingShingle | Candy Obdulia Sosa-Jimenez Homero Vladimir Rios-Figueroa Ana Luisa Solis-Gonzalez-Cosio A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician IEEE Access Mexican sign language depth sensor dynamic sign language recognition medicine consultation services sign language synthesis |
| title | A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician |
| title_full | A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician |
| title_fullStr | A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician |
| title_full_unstemmed | A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician |
| title_short | A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician |
| title_sort | prototype for mexican sign language recognition and synthesis in support of a primary care physician |
| topic | Mexican sign language depth sensor dynamic sign language recognition medicine consultation services sign language synthesis |
| url | https://ieeexplore.ieee.org/document/9970338/ |
| work_keys_str_mv | AT candyobduliasosajimenez aprototypeformexicansignlanguagerecognitionandsynthesisinsupportofaprimarycarephysician AT homerovladimirriosfigueroa aprototypeformexicansignlanguagerecognitionandsynthesisinsupportofaprimarycarephysician AT analuisasolisgonzalezcosio aprototypeformexicansignlanguagerecognitionandsynthesisinsupportofaprimarycarephysician AT candyobduliasosajimenez prototypeformexicansignlanguagerecognitionandsynthesisinsupportofaprimarycarephysician AT homerovladimirriosfigueroa prototypeformexicansignlanguagerecognitionandsynthesisinsupportofaprimarycarephysician AT analuisasolisgonzalezcosio prototypeformexicansignlanguagerecognitionandsynthesisinsupportofaprimarycarephysician |