Expressive Interaction Design Using Facial Muscles as Controllers
Here we describe a proof-of-concept case study focusing on the design and development of a novel computer interface that uses facial muscles to control interactivity within a virtual environment. We have developed a system comprised of skin-mounted electrodes that detect underlying muscle activity t...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Multimodal Technologies and Interaction |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2414-4088/6/9/78 |
| _version_ | 1827658232242372608 |
|---|---|
| author | Fiona French Cassandra Terry Saif Huq Isaac Furieri Mateusz Jarzembinski Simas Pauliukenas Neil Morrison Karen Shepherd |
| author_facet | Fiona French Cassandra Terry Saif Huq Isaac Furieri Mateusz Jarzembinski Simas Pauliukenas Neil Morrison Karen Shepherd |
| author_sort | Fiona French |
| collection | DOAJ |
| description | Here we describe a proof-of-concept case study focusing on the design and development of a novel computer interface that uses facial muscles to control interactivity within a virtual environment. We have developed a system comprised of skin-mounted electrodes that detect underlying muscle activity through electromyography. The signals from the electrodes are filtered and smoothed, then used as input data to an application that displays a virtual environment with a 3D animated avatar. The user’s expressions control the facial movements of the avatar, thus conveying user emotions through real-time animation of a representative face in a virtual scenario. To achieve this, we collaborated with our Public and Patient Involvement focus group to discuss concepts and design appropriate interactions, while simultaneously developing a prototype system. Programmers and 3D artists worked together to create a system whereby individual user facial muscles are connected to 3D animated models of the same muscle features represented in an avatar, providing the user with an option to receive visual and numerical feedback on the extent of their muscle control. Using the prototype system, people can communicate facial expressions virtually with each other, without the need for a camera. This research is part of an on-going project to develop a facial muscle rehabilitation system that can be optimized to help patients with conditions such as hypomimia. |
| first_indexed | 2024-03-09T22:58:52Z |
| format | Article |
| id | doaj.art-3447fa56386a4795abcde0989bcfc77b |
| institution | Directory Open Access Journal |
| issn | 2414-4088 |
| language | English |
| last_indexed | 2024-03-09T22:58:52Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Multimodal Technologies and Interaction |
| spelling | doaj.art-3447fa56386a4795abcde0989bcfc77b2023-11-23T18:05:06ZengMDPI AGMultimodal Technologies and Interaction2414-40882022-09-01697810.3390/mti6090078Expressive Interaction Design Using Facial Muscles as ControllersFiona French0Cassandra Terry1Saif Huq2Isaac Furieri3Mateusz Jarzembinski4Simas Pauliukenas5Neil Morrison6Karen Shepherd7Interaction Design Research Group, School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UKMolecular Systems for Health Research Group, School of Human Sciences, London Metropolitan University, London N7 8DB, UKInteraction Design Research Group, School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UKSchool of Computing and Digital Media, London Metropolitan University, London N7 8DB, UKSchool of Computing and Digital Media, London Metropolitan University, London N7 8DB, UKSchool of Computing and Digital Media, London Metropolitan University, London N7 8DB, UKNo affiliationNo affiliationHere we describe a proof-of-concept case study focusing on the design and development of a novel computer interface that uses facial muscles to control interactivity within a virtual environment. We have developed a system comprised of skin-mounted electrodes that detect underlying muscle activity through electromyography. The signals from the electrodes are filtered and smoothed, then used as input data to an application that displays a virtual environment with a 3D animated avatar. The user’s expressions control the facial movements of the avatar, thus conveying user emotions through real-time animation of a representative face in a virtual scenario. To achieve this, we collaborated with our Public and Patient Involvement focus group to discuss concepts and design appropriate interactions, while simultaneously developing a prototype system. Programmers and 3D artists worked together to create a system whereby individual user facial muscles are connected to 3D animated models of the same muscle features represented in an avatar, providing the user with an option to receive visual and numerical feedback on the extent of their muscle control. Using the prototype system, people can communicate facial expressions virtually with each other, without the need for a camera. This research is part of an on-going project to develop a facial muscle rehabilitation system that can be optimized to help patients with conditions such as hypomimia.https://www.mdpi.com/2414-4088/6/9/78interaction designinclusive designbiometricsfacial expression recognitionrehabilitationassistive technology |
| spellingShingle | Fiona French Cassandra Terry Saif Huq Isaac Furieri Mateusz Jarzembinski Simas Pauliukenas Neil Morrison Karen Shepherd Expressive Interaction Design Using Facial Muscles as Controllers Multimodal Technologies and Interaction interaction design inclusive design biometrics facial expression recognition rehabilitation assistive technology |
| title | Expressive Interaction Design Using Facial Muscles as Controllers |
| title_full | Expressive Interaction Design Using Facial Muscles as Controllers |
| title_fullStr | Expressive Interaction Design Using Facial Muscles as Controllers |
| title_full_unstemmed | Expressive Interaction Design Using Facial Muscles as Controllers |
| title_short | Expressive Interaction Design Using Facial Muscles as Controllers |
| title_sort | expressive interaction design using facial muscles as controllers |
| topic | interaction design inclusive design biometrics facial expression recognition rehabilitation assistive technology |
| url | https://www.mdpi.com/2414-4088/6/9/78 |
| work_keys_str_mv | AT fionafrench expressiveinteractiondesignusingfacialmusclesascontrollers AT cassandraterry expressiveinteractiondesignusingfacialmusclesascontrollers AT saifhuq expressiveinteractiondesignusingfacialmusclesascontrollers AT isaacfurieri expressiveinteractiondesignusingfacialmusclesascontrollers AT mateuszjarzembinski expressiveinteractiondesignusingfacialmusclesascontrollers AT simaspauliukenas expressiveinteractiondesignusingfacialmusclesascontrollers AT neilmorrison expressiveinteractiondesignusingfacialmusclesascontrollers AT karenshepherd expressiveinteractiondesignusingfacialmusclesascontrollers |