Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning
Online handwriting recognition has been the subject of research for many years. Despite that, a limited number of practical applications are currently available. The widespread use of devices such as smartphones, smartwatches, and tablets has not been enough to convince the user to use pen-based int...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/13/6707 |
| _version_ | 1797480840689614848 |
|---|---|
| author | Pedro Lopez-Rodriguez Juan Gabriel Avina-Cervantes Jose Luis Contreras-Hernandez Rodrigo Correa Jose Ruiz-Pinales |
| author_facet | Pedro Lopez-Rodriguez Juan Gabriel Avina-Cervantes Jose Luis Contreras-Hernandez Rodrigo Correa Jose Ruiz-Pinales |
| author_sort | Pedro Lopez-Rodriguez |
| collection | DOAJ |
| description | Online handwriting recognition has been the subject of research for many years. Despite that, a limited number of practical applications are currently available. The widespread use of devices such as smartphones, smartwatches, and tablets has not been enough to convince the user to use pen-based interfaces. This implies that more research on the pen interface and recognition methods is still necessary. This paper proposes a handwritten character recognition system based on 3D accelerometer signal processing using Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM). First, a user wearing an MYO armband on the forearm writes a multi-stroke freestyle character on a touchpad by using the finger or a pen. Next, the 3D accelerometer signals generated during the writing process are fed into a CNN, LSTM, or CNN-LSTM network for recognition. The convolutional backbone obtains spatial features in order to feed an LSTM that extracts short-term temporal information. The system was evaluated on a proprietary dataset of 3D accelerometer data collected from multiple users with an armband device, corresponding to handwritten English lowercase letters (a–z) and digits (0–9) in a freestyle. The results show that the proposed system overcomes other systems from the state of the art by 0.53%. |
| first_indexed | 2024-03-09T22:05:58Z |
| format | Article |
| id | doaj.art-ccdb5691d13945fcb29969ca8cf42293 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T22:05:58Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-ccdb5691d13945fcb29969ca8cf422932023-11-23T19:41:22ZengMDPI AGApplied Sciences2076-34172022-07-011213670710.3390/app12136707Handwriting Recognition Based on 3D Accelerometer Data by Deep LearningPedro Lopez-Rodriguez0Juan Gabriel Avina-Cervantes1Jose Luis Contreras-Hernandez2Rodrigo Correa3Jose Ruiz-Pinales4Digital Signal Processing and Telematics, Engineering Division of the Campus Irapuato-Salamanca (DICIS), Universidad de Guanajuato, Carr. Salamanca-Valle de Santiago Km 3.5 + 1.8, Palo Blanco, Salamanca 36885, MexicoDigital Signal Processing and Telematics, Engineering Division of the Campus Irapuato-Salamanca (DICIS), Universidad de Guanajuato, Carr. Salamanca-Valle de Santiago Km 3.5 + 1.8, Palo Blanco, Salamanca 36885, MexicoDigital Signal Processing and Telematics, Engineering Division of the Campus Irapuato-Salamanca (DICIS), Universidad de Guanajuato, Carr. Salamanca-Valle de Santiago Km 3.5 + 1.8, Palo Blanco, Salamanca 36885, MexicoEscuela de Ingenierías Eléctrica, Electrónica y de Telecomunicaciones, Universidad Industrial de Santander, Cra 27 Calle 9, Bucaramanga 680002, ColombiaDigital Signal Processing and Telematics, Engineering Division of the Campus Irapuato-Salamanca (DICIS), Universidad de Guanajuato, Carr. Salamanca-Valle de Santiago Km 3.5 + 1.8, Palo Blanco, Salamanca 36885, MexicoOnline handwriting recognition has been the subject of research for many years. Despite that, a limited number of practical applications are currently available. The widespread use of devices such as smartphones, smartwatches, and tablets has not been enough to convince the user to use pen-based interfaces. This implies that more research on the pen interface and recognition methods is still necessary. This paper proposes a handwritten character recognition system based on 3D accelerometer signal processing using Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM). First, a user wearing an MYO armband on the forearm writes a multi-stroke freestyle character on a touchpad by using the finger or a pen. Next, the 3D accelerometer signals generated during the writing process are fed into a CNN, LSTM, or CNN-LSTM network for recognition. The convolutional backbone obtains spatial features in order to feed an LSTM that extracts short-term temporal information. The system was evaluated on a proprietary dataset of 3D accelerometer data collected from multiple users with an armband device, corresponding to handwritten English lowercase letters (a–z) and digits (0–9) in a freestyle. The results show that the proposed system overcomes other systems from the state of the art by 0.53%.https://www.mdpi.com/2076-3417/12/13/67073D accelerometer datahandwritten character recognitionConvolutional Neural Networks (CNN)Long Short-Term Memory (LSTM)3D signal processing |
| spellingShingle | Pedro Lopez-Rodriguez Juan Gabriel Avina-Cervantes Jose Luis Contreras-Hernandez Rodrigo Correa Jose Ruiz-Pinales Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning Applied Sciences 3D accelerometer data handwritten character recognition Convolutional Neural Networks (CNN) Long Short-Term Memory (LSTM) 3D signal processing |
| title | Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning |
| title_full | Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning |
| title_fullStr | Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning |
| title_full_unstemmed | Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning |
| title_short | Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning |
| title_sort | handwriting recognition based on 3d accelerometer data by deep learning |
| topic | 3D accelerometer data handwritten character recognition Convolutional Neural Networks (CNN) Long Short-Term Memory (LSTM) 3D signal processing |
| url | https://www.mdpi.com/2076-3417/12/13/6707 |
| work_keys_str_mv | AT pedrolopezrodriguez handwritingrecognitionbasedon3daccelerometerdatabydeeplearning AT juangabrielavinacervantes handwritingrecognitionbasedon3daccelerometerdatabydeeplearning AT joseluiscontrerashernandez handwritingrecognitionbasedon3daccelerometerdatabydeeplearning AT rodrigocorrea handwritingrecognitionbasedon3daccelerometerdatabydeeplearning AT joseruizpinales handwritingrecognitionbasedon3daccelerometerdatabydeeplearning |