Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings
Brain–Machine Interfaces (BMIs) have made significant progress in recent years; however, there are still several application areas in which improvement is needed, including the accurate prediction of body movement during Virtual Reality (VR) simulations. To achieve a high level of immersion in VR se...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Virtual Worlds |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2813-2084/2/2/11 |
| _version_ | 1827735397633884160 |
|---|---|
| author | Jacob Kritikos Alexandros Makrypidis Aristomenis Alevizopoulos Georgios Alevizopoulos Dimitris Koutsouris |
| author_facet | Jacob Kritikos Alexandros Makrypidis Aristomenis Alevizopoulos Georgios Alevizopoulos Dimitris Koutsouris |
| author_sort | Jacob Kritikos |
| collection | DOAJ |
| description | Brain–Machine Interfaces (BMIs) have made significant progress in recent years; however, there are still several application areas in which improvement is needed, including the accurate prediction of body movement during Virtual Reality (VR) simulations. To achieve a high level of immersion in VR sessions, it is important to have bidirectional interaction, which is typically achieved through the use of movement-tracking devices, such as controllers and body sensors. However, it may be possible to eliminate the need for these external tracking devices by directly acquiring movement information from the motor cortex via electroencephalography (EEG) recordings. This could potentially lead to more seamless and immersive VR experiences. There have been numerous studies that have investigated EEG recordings during movement. While the majority of these studies have focused on movement prediction based on brain signals, a smaller number of them have focused on how to utilize them during VR simulations. This suggests that there is still a need for further research in this area in order to fully understand the potential for using EEG to predict movement in VR simulations. We propose two neural network decoders designed to predict pre-arm-movement and during-arm-movement behavior based on brain activity recorded during the execution of VR simulation tasks in this research. For both decoders, we employ a Long Short-Term Memory model. The study’s findings are highly encouraging, lending credence to the premise that this technology has the ability to replace external tracking devices. |
| first_indexed | 2024-03-11T01:50:22Z |
| format | Article |
| id | doaj.art-be12219f3add4e4da7bab9cace1aa383 |
| institution | Directory Open Access Journal |
| issn | 2813-2084 |
| language | English |
| last_indexed | 2024-03-11T01:50:22Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Virtual Worlds |
| spelling | doaj.art-be12219f3add4e4da7bab9cace1aa3832023-11-18T13:00:38ZengMDPI AGVirtual Worlds2813-20842023-06-012218220210.3390/virtualworlds2020011Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG RecordingsJacob Kritikos0Alexandros Makrypidis1Aristomenis Alevizopoulos2Georgios Alevizopoulos3Dimitris Koutsouris4Department of Bioengineering, Imperial College London, London SW7 2BX, UKDepartment of Bioengineering, Imperial College London, London SW7 2BX, UKSchool of Medicine, National and Kapodistrian University of Athens, 11527 Athens, GreecePsychiatric Clinic, Agioi Anargyroi General Oncological Hospital of Kifissia, 14564 Athens, GreeceSchool of Electrical and Computer Engineering, National Technical University of Athens, 15772 Athens, GreeceBrain–Machine Interfaces (BMIs) have made significant progress in recent years; however, there are still several application areas in which improvement is needed, including the accurate prediction of body movement during Virtual Reality (VR) simulations. To achieve a high level of immersion in VR sessions, it is important to have bidirectional interaction, which is typically achieved through the use of movement-tracking devices, such as controllers and body sensors. However, it may be possible to eliminate the need for these external tracking devices by directly acquiring movement information from the motor cortex via electroencephalography (EEG) recordings. This could potentially lead to more seamless and immersive VR experiences. There have been numerous studies that have investigated EEG recordings during movement. While the majority of these studies have focused on movement prediction based on brain signals, a smaller number of them have focused on how to utilize them during VR simulations. This suggests that there is still a need for further research in this area in order to fully understand the potential for using EEG to predict movement in VR simulations. We propose two neural network decoders designed to predict pre-arm-movement and during-arm-movement behavior based on brain activity recorded during the execution of VR simulation tasks in this research. For both decoders, we employ a Long Short-Term Memory model. The study’s findings are highly encouraging, lending credence to the premise that this technology has the ability to replace external tracking devices.https://www.mdpi.com/2813-2084/2/2/11Brain–Machine InterfaceelectroencephalographyVirtual Reality |
| spellingShingle | Jacob Kritikos Alexandros Makrypidis Aristomenis Alevizopoulos Georgios Alevizopoulos Dimitris Koutsouris Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings Virtual Worlds Brain–Machine Interface electroencephalography Virtual Reality |
| title | Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings |
| title_full | Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings |
| title_fullStr | Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings |
| title_full_unstemmed | Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings |
| title_short | Can Brain–Computer Interfaces Replace Virtual Reality Controllers? A Machine Learning Movement Prediction Model during Virtual Reality Simulation Using EEG Recordings |
| title_sort | can brain computer interfaces replace virtual reality controllers a machine learning movement prediction model during virtual reality simulation using eeg recordings |
| topic | Brain–Machine Interface electroencephalography Virtual Reality |
| url | https://www.mdpi.com/2813-2084/2/2/11 |
| work_keys_str_mv | AT jacobkritikos canbraincomputerinterfacesreplacevirtualrealitycontrollersamachinelearningmovementpredictionmodelduringvirtualrealitysimulationusingeegrecordings AT alexandrosmakrypidis canbraincomputerinterfacesreplacevirtualrealitycontrollersamachinelearningmovementpredictionmodelduringvirtualrealitysimulationusingeegrecordings AT aristomenisalevizopoulos canbraincomputerinterfacesreplacevirtualrealitycontrollersamachinelearningmovementpredictionmodelduringvirtualrealitysimulationusingeegrecordings AT georgiosalevizopoulos canbraincomputerinterfacesreplacevirtualrealitycontrollersamachinelearningmovementpredictionmodelduringvirtualrealitysimulationusingeegrecordings AT dimitriskoutsouris canbraincomputerinterfacesreplacevirtualrealitycontrollersamachinelearningmovementpredictionmodelduringvirtualrealitysimulationusingeegrecordings |