Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office
The paper addresses the recognition of dynamic Polish Sign Language expressions in an experimental system supporting deaf people in an office when applying for an ID card. A method of processing a continuous stream of RGB-D data and a feature vector are proposed. The classification is carried out us...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/8/2190 |
| _version_ | 1797570749007921152 |
|---|---|
| author | Tomasz Kapuscinski Marian Wysocki |
| author_facet | Tomasz Kapuscinski Marian Wysocki |
| author_sort | Tomasz Kapuscinski |
| collection | DOAJ |
| description | The paper addresses the recognition of dynamic Polish Sign Language expressions in an experimental system supporting deaf people in an office when applying for an ID card. A method of processing a continuous stream of RGB-D data and a feature vector are proposed. The classification is carried out using the k-nearest neighbors algorithm with dynamic time warping, hidden Markov models, and bidirectional long short-term memory. The leave-one-subject-out protocol is used for the dataset containing 121 Polish Sign Language sentences performed five times by four deaf people. A data augmentation method is also proposed and tested. Preliminary observations and conclusions from the use of the system in a laboratory, as well as in real conditions with an experimental installation in the Office of Civil Affairs are given. |
| first_indexed | 2024-03-10T20:29:48Z |
| format | Article |
| id | doaj.art-375f84f2d7a94ef7ac1dd11cdaae7b75 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T20:29:48Z |
| publishDate | 2020-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-375f84f2d7a94ef7ac1dd11cdaae7b752023-11-19T21:27:05ZengMDPI AGSensors1424-82202020-04-01208219010.3390/s20082190Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City OfficeTomasz Kapuscinski0Marian Wysocki1Department of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Rzeszow University of Technology, W. Pola 2, 35-959 Rzeszow, PolandDepartment of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Rzeszow University of Technology, W. Pola 2, 35-959 Rzeszow, PolandThe paper addresses the recognition of dynamic Polish Sign Language expressions in an experimental system supporting deaf people in an office when applying for an ID card. A method of processing a continuous stream of RGB-D data and a feature vector are proposed. The classification is carried out using the k-nearest neighbors algorithm with dynamic time warping, hidden Markov models, and bidirectional long short-term memory. The leave-one-subject-out protocol is used for the dataset containing 121 Polish Sign Language sentences performed five times by four deaf people. A data augmentation method is also proposed and tested. Preliminary observations and conclusions from the use of the system in a laboratory, as well as in real conditions with an experimental installation in the Office of Civil Affairs are given.https://www.mdpi.com/1424-8220/20/8/2190human–computer interfacecomputer visionsign language recognition |
| spellingShingle | Tomasz Kapuscinski Marian Wysocki Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office Sensors human–computer interface computer vision sign language recognition |
| title | Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office |
| title_full | Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office |
| title_fullStr | Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office |
| title_full_unstemmed | Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office |
| title_short | Recognition of Signed Expressions in an Experimental System Supporting Deaf Clients in the City Office |
| title_sort | recognition of signed expressions in an experimental system supporting deaf clients in the city office |
| topic | human–computer interface computer vision sign language recognition |
| url | https://www.mdpi.com/1424-8220/20/8/2190 |
| work_keys_str_mv | AT tomaszkapuscinski recognitionofsignedexpressionsinanexperimentalsystemsupportingdeafclientsinthecityoffice AT marianwysocki recognitionofsignedexpressionsinanexperimentalsystemsupportingdeafclientsinthecityoffice |