Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers
Electromyography (EMG) signals have been used in designing muscle-machine interfaces (MuMIs) for various applications, ranging from entertainment (EMG controlled games) to human assistance and human augmentation (EMG controlled prostheses and exoskeletons). For this, classical machine learning metho...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9851477/ |
| _version_ | 1797805110528573440 |
|---|---|
| author | Ricardo V. Godoy Anany Dwivedi Minas Liarokapis |
| author_facet | Ricardo V. Godoy Anany Dwivedi Minas Liarokapis |
| author_sort | Ricardo V. Godoy |
| collection | DOAJ |
| description | Electromyography (EMG) signals have been used in designing muscle-machine interfaces (MuMIs) for various applications, ranging from entertainment (EMG controlled games) to human assistance and human augmentation (EMG controlled prostheses and exoskeletons). For this, classical machine learning methods such as Random Forest (RF) models have been used to decode EMG signals. However, these methods depend on several stages of signal pre-processing and extraction of hand-crafted features so as to obtain the desired output. In this work, we propose EMG based frameworks for the decoding of object motions in the execution of dexterous, in-hand manipulation tasks using raw EMG signals input and two novel deep learning (DL) techniques called Temporal Multi-Channel Transformers and Vision Transformers. The results obtained are compared, in terms of accuracy and speed of decoding the motion, with RF-based models and Convolutional Neural Networks as a benchmark. The models are trained for 11 subjects in a motion-object specific and motion-object generic way, using the 10-fold cross-validation procedure. This study shows that the performance of MuMIs can be improved by employing DL-based models with raw myoelectric activations instead of developing DL or classic machine learning models with hand-crafted features. |
| first_indexed | 2024-03-13T05:47:10Z |
| format | Article |
| id | doaj.art-ba046d41f9784742863227ea99fd0a84 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-13T05:47:10Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-ba046d41f9784742863227ea99fd0a842023-06-13T20:08:51ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102022-01-01302207221610.1109/TNSRE.2022.31966229851477Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision TransformersRicardo V. Godoy0https://orcid.org/0000-0002-5323-9299Anany Dwivedi1https://orcid.org/0000-0003-3262-6676Minas Liarokapis2https://orcid.org/0000-0002-6016-1477Department of Mechanical and Mechatronics Engineering, New Dexterity Research Group, The University of Auckland, Auckland, New ZealandChair of Autonomous Systems and Mechatronics, Friedrich-Alexander-Universität Erlangen-Nôrnberg, Erlangen, GermanyDepartment of Mechanical and Mechatronics Engineering, New Dexterity Research Group, The University of Auckland, Auckland, New ZealandElectromyography (EMG) signals have been used in designing muscle-machine interfaces (MuMIs) for various applications, ranging from entertainment (EMG controlled games) to human assistance and human augmentation (EMG controlled prostheses and exoskeletons). For this, classical machine learning methods such as Random Forest (RF) models have been used to decode EMG signals. However, these methods depend on several stages of signal pre-processing and extraction of hand-crafted features so as to obtain the desired output. In this work, we propose EMG based frameworks for the decoding of object motions in the execution of dexterous, in-hand manipulation tasks using raw EMG signals input and two novel deep learning (DL) techniques called Temporal Multi-Channel Transformers and Vision Transformers. The results obtained are compared, in terms of accuracy and speed of decoding the motion, with RF-based models and Convolutional Neural Networks as a benchmark. The models are trained for 11 subjects in a motion-object specific and motion-object generic way, using the 10-fold cross-validation procedure. This study shows that the performance of MuMIs can be improved by employing DL-based models with raw myoelectric activations instead of developing DL or classic machine learning models with hand-crafted features.https://ieeexplore.ieee.org/document/9851477/Electromyographymotion decodingdexterous manipulationdeep learningtransformers |
| spellingShingle | Ricardo V. Godoy Anany Dwivedi Minas Liarokapis Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers IEEE Transactions on Neural Systems and Rehabilitation Engineering Electromyography motion decoding dexterous manipulation deep learning transformers |
| title | Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers |
| title_full | Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers |
| title_fullStr | Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers |
| title_full_unstemmed | Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers |
| title_short | Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers |
| title_sort | electromyography based decoding of dexterous in hand manipulation motions with temporal multichannel vision transformers |
| topic | Electromyography motion decoding dexterous manipulation deep learning transformers |
| url | https://ieeexplore.ieee.org/document/9851477/ |
| work_keys_str_mv | AT ricardovgodoy electromyographybaseddecodingofdexterousinhandmanipulationmotionswithtemporalmultichannelvisiontransformers AT ananydwivedi electromyographybaseddecodingofdexterousinhandmanipulationmotionswithtemporalmultichannelvisiontransformers AT minasliarokapis electromyographybaseddecodingofdexterousinhandmanipulationmotionswithtemporalmultichannelvisiontransformers |