Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality
The increased use of Virtual and Augmented Reality based systems necessitates the development of more intuitive and unobtrusive means of interfacing. Over the last years, Electromyography (EMG) based interfaces have been employed for interaction with robotic and computer applications, but no studies...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9363902/ |
| _version_ | 1818351911973158912 |
|---|---|
| author | Yongje Kwon Anany Dwivedi Andrew J. McDaid Minas Liarokapis |
| author_facet | Yongje Kwon Anany Dwivedi Andrew J. McDaid Minas Liarokapis |
| author_sort | Yongje Kwon |
| collection | DOAJ |
| description | The increased use of Virtual and Augmented Reality based systems necessitates the development of more intuitive and unobtrusive means of interfacing. Over the last years, Electromyography (EMG) based interfaces have been employed for interaction with robotic and computer applications, but no studies have been carried out to investigate the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we compare the object motion decoding accuracy of an EMG based learning framework for two different dexterous manipulation scenarios: i) for simulated objects handled by a teleoperated model of a hand within a virtual environment and ii) for real, everyday life objects manipulated by the human hand. To do that, we utilize EMG activations from 16 muscle sites (9 on the hand and 7 on the forearm). The object motion decoding is formulated as a regression problem using the Random Forests methodology. A 5-fold cross validation procedure is used for model assessment purposes and the feature variable importance values are calculated for each model. The decoding accuracy for the real world is considerably higher than the virtual world. Each of the objects examined had a single manipulation motion that offered the highest estimation accuracy across both worlds. This study also shows that it is feasible to decode the object motions using just the myoelectric activations of the muscles of the forearm and the hand. This is particularly surprising since simulations lacked haptic feedback and the ability to account for other dynamic phenomena like friction and contact rolling. |
| first_indexed | 2024-12-13T18:45:16Z |
| format | Article |
| id | doaj.art-c6d54d8ed0654c6aab21808caa66d9a9 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T18:45:16Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c6d54d8ed0654c6aab21808caa66d9a92022-12-21T23:35:05ZengIEEEIEEE Access2169-35362021-01-019372973731010.1109/ACCESS.2021.30623649363902Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual RealityYongje Kwon0https://orcid.org/0000-0001-9285-2734Anany Dwivedi1https://orcid.org/0000-0003-3262-6676Andrew J. McDaid2https://orcid.org/0000-0003-3316-7344Minas Liarokapis3https://orcid.org/0000-0002-6016-1477Department of Mechanical Engineering, New Dexterity Research Group, The University of Auckland, Auckland, New ZealandDepartment of Mechanical Engineering, New Dexterity Research Group, The University of Auckland, Auckland, New ZealandDepartment of Mechanical Engineering, The University of Auckland, Auckland, New ZealandDepartment of Mechanical Engineering, New Dexterity Research Group, The University of Auckland, Auckland, New ZealandThe increased use of Virtual and Augmented Reality based systems necessitates the development of more intuitive and unobtrusive means of interfacing. Over the last years, Electromyography (EMG) based interfaces have been employed for interaction with robotic and computer applications, but no studies have been carried out to investigate the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we compare the object motion decoding accuracy of an EMG based learning framework for two different dexterous manipulation scenarios: i) for simulated objects handled by a teleoperated model of a hand within a virtual environment and ii) for real, everyday life objects manipulated by the human hand. To do that, we utilize EMG activations from 16 muscle sites (9 on the hand and 7 on the forearm). The object motion decoding is formulated as a regression problem using the Random Forests methodology. A 5-fold cross validation procedure is used for model assessment purposes and the feature variable importance values are calculated for each model. The decoding accuracy for the real world is considerably higher than the virtual world. Each of the objects examined had a single manipulation motion that offered the highest estimation accuracy across both worlds. This study also shows that it is feasible to decode the object motions using just the myoelectric activations of the muscles of the forearm and the hand. This is particularly surprising since simulations lacked haptic feedback and the ability to account for other dynamic phenomena like friction and contact rolling.https://ieeexplore.ieee.org/document/9363902/Electromyographyvirtual realitymuscle computer interfacesmuscle machine interfacesmachine learning |
| spellingShingle | Yongje Kwon Anany Dwivedi Andrew J. McDaid Minas Liarokapis Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality IEEE Access Electromyography virtual reality muscle computer interfaces muscle machine interfaces machine learning |
| title | Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality |
| title_full | Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality |
| title_fullStr | Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality |
| title_full_unstemmed | Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality |
| title_short | Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality |
| title_sort | electromyography based decoding of dexterous in hand manipulation of objects comparing task execution in real world and virtual reality |
| topic | Electromyography virtual reality muscle computer interfaces muscle machine interfaces machine learning |
| url | https://ieeexplore.ieee.org/document/9363902/ |
| work_keys_str_mv | AT yongjekwon electromyographybaseddecodingofdexterousinhandmanipulationofobjectscomparingtaskexecutioninrealworldandvirtualreality AT ananydwivedi electromyographybaseddecodingofdexterousinhandmanipulationofobjectscomparingtaskexecutioninrealworldandvirtualreality AT andrewjmcdaid electromyographybaseddecodingofdexterousinhandmanipulationofobjectscomparingtaskexecutioninrealworldandvirtualreality AT minasliarokapis electromyographybaseddecodingofdexterousinhandmanipulationofobjectscomparingtaskexecutioninrealworldandvirtualreality |