Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals
Recognizing applied hand forces using force myography (FMG) biosignals requires adequate training data to facilitate physical human-robot interactions (pHRI). But in practice, data is often scarce, and labels are usually unavailable or time consuming to generate. Synthesizing FMG biosignals can be a...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9810260/ |
| _version_ | 1828303164065972224 |
|---|---|
| author | Umme Zakia Arnab Barua Xianta Jiang Carlo Menon |
| author_facet | Umme Zakia Arnab Barua Xianta Jiang Carlo Menon |
| author_sort | Umme Zakia |
| collection | DOAJ |
| description | Recognizing applied hand forces using force myography (FMG) biosignals requires adequate training data to facilitate physical human-robot interactions (pHRI). But in practice, data is often scarce, and labels are usually unavailable or time consuming to generate. Synthesizing FMG biosignals can be a viable solution. Therefore, in this paper, we propose for the first time a dual-phased algorithm based on semi-supervised adversarial learning utilizing fewer labeled real FMG data with generated unlabeled synthetic FMG data. We conducted a pilot study to test this algorithm in estimating applied forces during interactions with a Kuka robot in 1D-X, Y, Z directions. Initially, an unsupervised FMG-based deep convolutional generative adversarial network (FMG-DCGAN) model was employed to generate real-like synthetic FMG data. A variety of transformation functions were used to observe domain randomization for increasing data variability and for representing authentic physiological, environmental changes. Cosine similarity score and generated-to-input-data ratio were used as decision criteria minimizing the reality gap between real and synthetic data and helped avoid risks associated with wrong predictions. Finally, the FMG-DCGAN model was pretrained to generate pseudo-labels for unlabeled real and synthetic data, further retrained using all labeled and pseudo-labeled data and was termed as the self-trained FMG-DCGAN model. Lastly, this model was evaluated on unseen real test data and achieved accuracies of 85%>R<sup>2</sup> > 77% in force estimation compared to the corresponding supervised baseline model (89%>R<sup>2</sup> > 78%). Therefore, the proposed method can be more practical for use in FMG-based HRI, rehabilitation, and prosthetic control for daily, repetitive usage even with few labeled data. |
| first_indexed | 2024-04-13T13:47:40Z |
| format | Article |
| id | doaj.art-5d659fd0656848c783e296ed63691298 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-13T13:47:40Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5d659fd0656848c783e296ed636912982022-12-22T02:44:25ZengIEEEIEEE Access2169-35362022-01-0110699106992110.1109/ACCESS.2022.31871159810260Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography SignalsUmme Zakia0https://orcid.org/0000-0002-7313-3969Arnab Barua1Xianta Jiang2https://orcid.org/0000-0002-3219-1871Carlo Menon3https://orcid.org/0000-0002-2309-9977Menrva Research Group, School of Mechatronic Systems Engineering, Simon Fraser University, Burnaby, BC, CanadaDepartment of Computer Science, Memorial University of Newfoundland, St. John’s, NL, CanadaDepartment of Computer Science, Memorial University of Newfoundland, St. John’s, NL, CanadaMenrva Research Group, School of Mechatronic Systems Engineering, Simon Fraser University, Burnaby, BC, CanadaRecognizing applied hand forces using force myography (FMG) biosignals requires adequate training data to facilitate physical human-robot interactions (pHRI). But in practice, data is often scarce, and labels are usually unavailable or time consuming to generate. Synthesizing FMG biosignals can be a viable solution. Therefore, in this paper, we propose for the first time a dual-phased algorithm based on semi-supervised adversarial learning utilizing fewer labeled real FMG data with generated unlabeled synthetic FMG data. We conducted a pilot study to test this algorithm in estimating applied forces during interactions with a Kuka robot in 1D-X, Y, Z directions. Initially, an unsupervised FMG-based deep convolutional generative adversarial network (FMG-DCGAN) model was employed to generate real-like synthetic FMG data. A variety of transformation functions were used to observe domain randomization for increasing data variability and for representing authentic physiological, environmental changes. Cosine similarity score and generated-to-input-data ratio were used as decision criteria minimizing the reality gap between real and synthetic data and helped avoid risks associated with wrong predictions. Finally, the FMG-DCGAN model was pretrained to generate pseudo-labels for unlabeled real and synthetic data, further retrained using all labeled and pseudo-labeled data and was termed as the self-trained FMG-DCGAN model. Lastly, this model was evaluated on unseen real test data and achieved accuracies of 85%>R<sup>2</sup> > 77% in force estimation compared to the corresponding supervised baseline model (89%>R<sup>2</sup> > 78%). Therefore, the proposed method can be more practical for use in FMG-based HRI, rehabilitation, and prosthetic control for daily, repetitive usage even with few labeled data.https://ieeexplore.ieee.org/document/9810260/FMG signalapplied force estimationunsupervised learninggenerative adversarial networksdomain randomizationtransformation functions |
| spellingShingle | Umme Zakia Arnab Barua Xianta Jiang Carlo Menon Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals IEEE Access FMG signal applied force estimation unsupervised learning generative adversarial networks domain randomization transformation functions |
| title | Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals |
| title_full | Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals |
| title_fullStr | Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals |
| title_full_unstemmed | Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals |
| title_short | Unsupervised, Semi-Supervised Interactive Force Estimations During pHRI via Generated Synthetic Force Myography Signals |
| title_sort | unsupervised semi supervised interactive force estimations during phri via generated synthetic force myography signals |
| topic | FMG signal applied force estimation unsupervised learning generative adversarial networks domain randomization transformation functions |
| url | https://ieeexplore.ieee.org/document/9810260/ |
| work_keys_str_mv | AT ummezakia unsupervisedsemisupervisedinteractiveforceestimationsduringphriviageneratedsyntheticforcemyographysignals AT arnabbarua unsupervisedsemisupervisedinteractiveforceestimationsduringphriviageneratedsyntheticforcemyographysignals AT xiantajiang unsupervisedsemisupervisedinteractiveforceestimationsduringphriviageneratedsyntheticforcemyographysignals AT carlomenon unsupervisedsemisupervisedinteractiveforceestimationsduringphriviageneratedsyntheticforcemyographysignals |