Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation
Haptic guidance is a promising method for assisting an operator in solving robotic remote operation tasks. It can be implemented through different methods, such as virtual fixtures, where a predefined trajectory is used to generate guidance forces, or interactive guidance, where sensor measurements...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-07-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/8/7/772 |
| _version_ | 1798038973798416384 |
|---|---|
| author | Carlos J. Pérez-del-Pulgar Jan Smisek Irene Rivas-Blanco Andre Schiele Victor F. Muñoz |
| author_facet | Carlos J. Pérez-del-Pulgar Jan Smisek Irene Rivas-Blanco Andre Schiele Victor F. Muñoz |
| author_sort | Carlos J. Pérez-del-Pulgar |
| collection | DOAJ |
| description | Haptic guidance is a promising method for assisting an operator in solving robotic remote operation tasks. It can be implemented through different methods, such as virtual fixtures, where a predefined trajectory is used to generate guidance forces, or interactive guidance, where sensor measurements are used to assist the operator in real-time. During the last years, the use of learning from demonstration (LfD) has been proposed to perform interactive guidance based on simple tasks that are usually composed of a single stage. However, it would be desirable to improve this approach to solve complex tasks composed of several stages or gestures. This paper extends the LfD approach for object telemanipulation where the task to be solved is divided into a set of gestures that need to be detected. Thus, each gesture is previously trained and encoded within a Gaussian mixture model using LfD, and stored in a gesture library. During telemanipulation, depending on the sensory information, the gesture that is being carried out is recognized using the same LfD trained model for haptic guidance. The method was experimentally verified in a teleoperated peg-in-hole insertion task. A KUKA LWR4+ lightweight robot was remotely controlled with a Sigma.7 haptic device with LfD-based shared control. Finally, a comparison was carried out to evaluate the performance of Gaussian mixture models with a well-established gesture recognition method, continuous hidden Markov models, for the same task. Results show that the Gaussian mixture models (GMM)-based method slightly improves the success rate, with lower training and recognition processing times. |
| first_indexed | 2024-04-11T21:47:32Z |
| format | Article |
| id | doaj.art-2be8234ad6154c8781a8e9decf4e1c9d |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-04-11T21:47:32Z |
| publishDate | 2019-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-2be8234ad6154c8781a8e9decf4e1c9d2022-12-22T04:01:21ZengMDPI AGElectronics2079-92922019-07-018777210.3390/electronics8070772electronics8070772Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided TelemanipulationCarlos J. Pérez-del-Pulgar0Jan Smisek1Irene Rivas-Blanco2Andre Schiele3Victor F. Muñoz4Department of Systems Engineering and Automation, Universidad de Málaga, Andalucía Tech, 29071 Málaga, SpainDelft University of Technology, 2628 Delft, The NetherlandsDepartment of Systems Engineering and Automation, Universidad de Málaga, Andalucía Tech, 29071 Málaga, SpainDelft University of Technology, 2628 Delft, The NetherlandsDepartment of Systems Engineering and Automation, Universidad de Málaga, Andalucía Tech, 29071 Málaga, SpainHaptic guidance is a promising method for assisting an operator in solving robotic remote operation tasks. It can be implemented through different methods, such as virtual fixtures, where a predefined trajectory is used to generate guidance forces, or interactive guidance, where sensor measurements are used to assist the operator in real-time. During the last years, the use of learning from demonstration (LfD) has been proposed to perform interactive guidance based on simple tasks that are usually composed of a single stage. However, it would be desirable to improve this approach to solve complex tasks composed of several stages or gestures. This paper extends the LfD approach for object telemanipulation where the task to be solved is divided into a set of gestures that need to be detected. Thus, each gesture is previously trained and encoded within a Gaussian mixture model using LfD, and stored in a gesture library. During telemanipulation, depending on the sensory information, the gesture that is being carried out is recognized using the same LfD trained model for haptic guidance. The method was experimentally verified in a teleoperated peg-in-hole insertion task. A KUKA LWR4+ lightweight robot was remotely controlled with a Sigma.7 haptic device with LfD-based shared control. Finally, a comparison was carried out to evaluate the performance of Gaussian mixture models with a well-established gesture recognition method, continuous hidden Markov models, for the same task. Results show that the Gaussian mixture models (GMM)-based method slightly improves the success rate, with lower training and recognition processing times.https://www.mdpi.com/2079-9292/8/7/772roboticstelemanipulationhapticsmachine learninggesture recognition |
| spellingShingle | Carlos J. Pérez-del-Pulgar Jan Smisek Irene Rivas-Blanco Andre Schiele Victor F. Muñoz Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation Electronics robotics telemanipulation haptics machine learning gesture recognition |
| title | Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation |
| title_full | Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation |
| title_fullStr | Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation |
| title_full_unstemmed | Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation |
| title_short | Using Gaussian Mixture Models for Gesture Recognition During Haptically Guided Telemanipulation |
| title_sort | using gaussian mixture models for gesture recognition during haptically guided telemanipulation |
| topic | robotics telemanipulation haptics machine learning gesture recognition |
| url | https://www.mdpi.com/2079-9292/8/7/772 |
| work_keys_str_mv | AT carlosjperezdelpulgar usinggaussianmixturemodelsforgesturerecognitionduringhapticallyguidedtelemanipulation AT jansmisek usinggaussianmixturemodelsforgesturerecognitionduringhapticallyguidedtelemanipulation AT irenerivasblanco usinggaussianmixturemodelsforgesturerecognitionduringhapticallyguidedtelemanipulation AT andreschiele usinggaussianmixturemodelsforgesturerecognitionduringhapticallyguidedtelemanipulation AT victorfmunoz usinggaussianmixturemodelsforgesturerecognitionduringhapticallyguidedtelemanipulation |