An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion
For online sign language recognition (SLR) based on inertial measurement unit (IMU) and a surface electromyography (sEMG) sensor, achieving high-accuracy is a major challenge. The traditional method for that is the segmentation−recognition mechanism, which has two key challenges: (1) it is...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-05-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/11/2495 |
| _version_ | 1811300039449903104 |
|---|---|
| author | Fei Wang Shusen Zhao Xingqun Zhou Chen Li Mingyao Li Zhen Zeng |
| author_facet | Fei Wang Shusen Zhao Xingqun Zhou Chen Li Mingyao Li Zhen Zeng |
| author_sort | Fei Wang |
| collection | DOAJ |
| description | For online sign language recognition (SLR) based on inertial measurement unit (IMU) and a surface electromyography (sEMG) sensor, achieving high-accuracy is a major challenge. The traditional method for that is the segmentation−recognition mechanism, which has two key challenges: (1) it is difficult to design a highly robust segmentation method for online data with inconspicuous segmentation information; and (2) the diversity of input data will increase the burden of the classification. The recognition−verification mechanism was proposed to improve the performance of online SLR. In the recognition stage, we used sliding windows to pull the data, and applied a convolutional neural network (CNN) to classify the sign language signal. In the verification stage, the confidence was evaluated by the Siamese network to judge the correctness of the classification results. The accuracy and rapidity of the classification model were discussed for 86 categories of Chinese sign language. In the experiments for online SLR, the superiority of the recognition−verification mechanism compared to the segmentation−recognition mechanism was verified. |
| first_indexed | 2024-04-13T06:45:22Z |
| format | Article |
| id | doaj.art-0bfb71dfa58c4b11b01da8458943ebc9 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-13T06:45:22Z |
| publishDate | 2019-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-0bfb71dfa58c4b11b01da8458943ebc92022-12-22T02:57:35ZengMDPI AGSensors1424-82202019-05-011911249510.3390/s19112495s19112495An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information FusionFei Wang0Shusen Zhao1Xingqun Zhou2Chen Li3Mingyao Li4Zhen Zeng5Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, ChinaFaculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, ChinaCollege of Information Science and Engineering, Northeastern University, Shenyang 110004, ChinaFaculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, ChinaFaculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, ChinaCollege of Information Science and Engineering, Northeastern University, Shenyang 110004, ChinaFor online sign language recognition (SLR) based on inertial measurement unit (IMU) and a surface electromyography (sEMG) sensor, achieving high-accuracy is a major challenge. The traditional method for that is the segmentation−recognition mechanism, which has two key challenges: (1) it is difficult to design a highly robust segmentation method for online data with inconspicuous segmentation information; and (2) the diversity of input data will increase the burden of the classification. The recognition−verification mechanism was proposed to improve the performance of online SLR. In the recognition stage, we used sliding windows to pull the data, and applied a convolutional neural network (CNN) to classify the sign language signal. In the verification stage, the confidence was evaluated by the Siamese network to judge the correctness of the classification results. The accuracy and rapidity of the classification model were discussed for 86 categories of Chinese sign language. In the experiments for online SLR, the superiority of the recognition−verification mechanism compared to the segmentation−recognition mechanism was verified.https://www.mdpi.com/1424-8220/19/11/2495sign language recognition (SLR)recognition–verification mechanismsurface electromyography (sEMG)CNNSiamese networkVGG |
| spellingShingle | Fei Wang Shusen Zhao Xingqun Zhou Chen Li Mingyao Li Zhen Zeng An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion Sensors sign language recognition (SLR) recognition–verification mechanism surface electromyography (sEMG) CNN Siamese network VGG |
| title | An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion |
| title_full | An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion |
| title_fullStr | An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion |
| title_full_unstemmed | An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion |
| title_short | An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion |
| title_sort | recognition verification mechanism for real time chinese sign language recognition based on multi information fusion |
| topic | sign language recognition (SLR) recognition–verification mechanism surface electromyography (sEMG) CNN Siamese network VGG |
| url | https://www.mdpi.com/1424-8220/19/11/2495 |
| work_keys_str_mv | AT feiwang anrecognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT shusenzhao anrecognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT xingqunzhou anrecognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT chenli anrecognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT mingyaoli anrecognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT zhenzeng anrecognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT feiwang recognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT shusenzhao recognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT xingqunzhou recognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT chenli recognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT mingyaoli recognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion AT zhenzeng recognitionverificationmechanismforrealtimechinesesignlanguagerecognitionbasedonmultiinformationfusion |