IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography
Deaf and hearing-impaired people always face communication barriers. Non-invasive surface electromyography (sEMG) sensor-based sign language recognition (SLR) technology can help them to better integrate into social life. Since the traditional tandem convolutional neural network (CNN) structure used...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/13/5775 |
| _version_ | 1797590841456328704 |
|---|---|
| author | Xiangrui Wang Lu Tang Qibin Zheng Xilin Yang Zhiyuan Lu |
| author_facet | Xiangrui Wang Lu Tang Qibin Zheng Xilin Yang Zhiyuan Lu |
| author_sort | Xiangrui Wang |
| collection | DOAJ |
| description | Deaf and hearing-impaired people always face communication barriers. Non-invasive surface electromyography (sEMG) sensor-based sign language recognition (SLR) technology can help them to better integrate into social life. Since the traditional tandem convolutional neural network (CNN) structure used in most CNN-based studies inadequately captures the features of the input data, we propose a novel inception architecture with a residual module and dilated convolution (IRDC-net) to enlarge the receptive fields and enrich the feature maps, applying it to SLR tasks for the first time. This work first transformed the time domain signal into a time–frequency domain using discrete Fourier transformation. Second, an IRDC-net was constructed to recognize ten Chinese sign language signs. Third, the tandem CNN networks VGG-net and ResNet-18 were compared with our proposed parallel structure network, IRDC-net. Finally, the public dataset Ninapro DB1 was utilized to verify the generalization performance of the IRDC-net. The results showed that after transforming the time domain sEMG signal into the time–frequency domain, the classification accuracy (acc) increased from 84.29% to 91.70% when using the IRDC-net on our sign language dataset. Furthermore, for the time–frequency information of the public dataset Ninapro DB1, the classification accuracy reached 89.82%; this value is higher than that achieved in other recent studies. As such, our findings contribute to research into SLR tasks and to improving deaf and hearing-impaired people’s daily lives. |
| first_indexed | 2024-03-11T01:29:12Z |
| format | Article |
| id | doaj.art-3b2b3373faec4f0bbd5c30335b2b2465 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T01:29:12Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-3b2b3373faec4f0bbd5c30335b2b24652023-11-18T17:26:35ZengMDPI AGSensors1424-82202023-06-012313577510.3390/s23135775IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface ElectromyographyXiangrui Wang0Lu Tang1Qibin Zheng2Xilin Yang3Zhiyuan Lu4School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaSchool of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaSchool of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaSchool of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaSchool of Rehabilitation Science and Engineering, University of Health and Rehabilitation Sciences, Qingdao 266072, ChinaDeaf and hearing-impaired people always face communication barriers. Non-invasive surface electromyography (sEMG) sensor-based sign language recognition (SLR) technology can help them to better integrate into social life. Since the traditional tandem convolutional neural network (CNN) structure used in most CNN-based studies inadequately captures the features of the input data, we propose a novel inception architecture with a residual module and dilated convolution (IRDC-net) to enlarge the receptive fields and enrich the feature maps, applying it to SLR tasks for the first time. This work first transformed the time domain signal into a time–frequency domain using discrete Fourier transformation. Second, an IRDC-net was constructed to recognize ten Chinese sign language signs. Third, the tandem CNN networks VGG-net and ResNet-18 were compared with our proposed parallel structure network, IRDC-net. Finally, the public dataset Ninapro DB1 was utilized to verify the generalization performance of the IRDC-net. The results showed that after transforming the time domain sEMG signal into the time–frequency domain, the classification accuracy (acc) increased from 84.29% to 91.70% when using the IRDC-net on our sign language dataset. Furthermore, for the time–frequency information of the public dataset Ninapro DB1, the classification accuracy reached 89.82%; this value is higher than that achieved in other recent studies. As such, our findings contribute to research into SLR tasks and to improving deaf and hearing-impaired people’s daily lives.https://www.mdpi.com/1424-8220/23/13/5775sign language recognitionsurface electromyograminception networkresidual moduledilated convolution |
| spellingShingle | Xiangrui Wang Lu Tang Qibin Zheng Xilin Yang Zhiyuan Lu IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography Sensors sign language recognition surface electromyogram inception network residual module dilated convolution |
| title | IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography |
| title_full | IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography |
| title_fullStr | IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography |
| title_full_unstemmed | IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography |
| title_short | IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography |
| title_sort | irdc net an inception network with a residual module and dilated convolution for sign language recognition based on surface electromyography |
| topic | sign language recognition surface electromyogram inception network residual module dilated convolution |
| url | https://www.mdpi.com/1424-8220/23/13/5775 |
| work_keys_str_mv | AT xiangruiwang irdcnetaninceptionnetworkwitharesidualmoduleanddilatedconvolutionforsignlanguagerecognitionbasedonsurfaceelectromyography AT lutang irdcnetaninceptionnetworkwitharesidualmoduleanddilatedconvolutionforsignlanguagerecognitionbasedonsurfaceelectromyography AT qibinzheng irdcnetaninceptionnetworkwitharesidualmoduleanddilatedconvolutionforsignlanguagerecognitionbasedonsurfaceelectromyography AT xilinyang irdcnetaninceptionnetworkwitharesidualmoduleanddilatedconvolutionforsignlanguagerecognitionbasedonsurfaceelectromyography AT zhiyuanlu irdcnetaninceptionnetworkwitharesidualmoduleanddilatedconvolutionforsignlanguagerecognitionbasedonsurfaceelectromyography |