BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation
While numerous studies show that brain signals contain information about an individual’s current state that are potentially valuable for smoothing man–machine interfaces, this has not yet lead to the use of brain computer interfaces (BCI) in daily life. One of the main challenges is the common requi...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-10-01
|
| Series: | Frontiers in Human Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnhum.2018.00420/full |
| _version_ | 1819057536886636544 |
|---|---|
| author | Anne-Marie Brouwer Jasper van der Waa Hans Stokking |
| author_facet | Anne-Marie Brouwer Jasper van der Waa Hans Stokking |
| author_sort | Anne-Marie Brouwer |
| collection | DOAJ |
| description | While numerous studies show that brain signals contain information about an individual’s current state that are potentially valuable for smoothing man–machine interfaces, this has not yet lead to the use of brain computer interfaces (BCI) in daily life. One of the main challenges is the common requirement of personal data that is correctly labeled concerning the state of interest in order to train a model, where this trained model is not guaranteed to generalize across time and context. Another challenge is the requirement to wear electrodes on the head. We here propose a BCI that can tackle these issues and may be a promising case for BCI research and application in everyday life. The BCI uses EEG signals to predict head rotation in order to improve images presented in a virtual reality (VR) headset. When presenting a 360° video to a headset, field-of-view approaches only stream the content that is in the current field of view and leave out the rest. When the user rotates the head, other content parts need to be made available soon enough to go unnoticed by the user, which is problematic given the available bandwidth. By predicting head rotation, the content parts adjacent to the currently viewed part could be retrieved in time for display when the rotation actually takes place. We here studied whether head rotations can be predicted on the basis of EEG sensor data and if so, whether application of such predictions could be applied to improve display of streaming images. Eleven participants generated left- and rightward head rotations while head movements were recorded using the headsets motion sensing system and EEG. We trained neural network models to distinguish EEG epochs preceding rightward, leftward, and no rotation. Applying these models to streaming EEG data that was withheld from the training showed that 400 ms before rotation onset, the probability “no rotation” started to decrease and the probabilities of an upcoming right- or leftward rotation started to diverge in the correct direction. In the proposed BCI scenario, users already wear a device on their head allowing for integrated EEG sensors. Moreover, it is possible to acquire accurately labeled training data on the fly, and continuously monitor and improve the model’s performance. The BCI can be harnessed if it will improve imagery and therewith enhance immersive experience. |
| first_indexed | 2024-12-21T13:40:52Z |
| format | Article |
| id | doaj.art-fe33adf361d14cf1b4ee32f7ec5d2789 |
| institution | Directory Open Access Journal |
| issn | 1662-5161 |
| language | English |
| last_indexed | 2024-12-21T13:40:52Z |
| publishDate | 2018-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Human Neuroscience |
| spelling | doaj.art-fe33adf361d14cf1b4ee32f7ec5d27892022-12-21T19:02:02ZengFrontiers Media S.A.Frontiers in Human Neuroscience1662-51612018-10-011210.3389/fnhum.2018.00420361578BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head RotationAnne-Marie Brouwer0Jasper van der Waa1Hans Stokking2Department of Perceptual and Cognitive Systems, Netherlands Organization for Applied Scientific Research (TNO), Soesterberg, NetherlandsDepartment of Perceptual and Cognitive Systems, Netherlands Organization for Applied Scientific Research (TNO), Soesterberg, NetherlandsDepartment of Media Networking, Netherlands Organization for Applied Scientific Research (TNO), Den Haag, NetherlandsWhile numerous studies show that brain signals contain information about an individual’s current state that are potentially valuable for smoothing man–machine interfaces, this has not yet lead to the use of brain computer interfaces (BCI) in daily life. One of the main challenges is the common requirement of personal data that is correctly labeled concerning the state of interest in order to train a model, where this trained model is not guaranteed to generalize across time and context. Another challenge is the requirement to wear electrodes on the head. We here propose a BCI that can tackle these issues and may be a promising case for BCI research and application in everyday life. The BCI uses EEG signals to predict head rotation in order to improve images presented in a virtual reality (VR) headset. When presenting a 360° video to a headset, field-of-view approaches only stream the content that is in the current field of view and leave out the rest. When the user rotates the head, other content parts need to be made available soon enough to go unnoticed by the user, which is problematic given the available bandwidth. By predicting head rotation, the content parts adjacent to the currently viewed part could be retrieved in time for display when the rotation actually takes place. We here studied whether head rotations can be predicted on the basis of EEG sensor data and if so, whether application of such predictions could be applied to improve display of streaming images. Eleven participants generated left- and rightward head rotations while head movements were recorded using the headsets motion sensing system and EEG. We trained neural network models to distinguish EEG epochs preceding rightward, leftward, and no rotation. Applying these models to streaming EEG data that was withheld from the training showed that 400 ms before rotation onset, the probability “no rotation” started to decrease and the probabilities of an upcoming right- or leftward rotation started to diverge in the correct direction. In the proposed BCI scenario, users already wear a device on their head allowing for integrated EEG sensors. Moreover, it is possible to acquire accurately labeled training data on the fly, and continuously monitor and improve the model’s performance. The BCI can be harnessed if it will improve imagery and therewith enhance immersive experience.https://www.frontiersin.org/article/10.3389/fnhum.2018.00420/fullEEGbrain computer interfaceneuroadaptive technologyvirtual realityhead mounted displayhead rotation |
| spellingShingle | Anne-Marie Brouwer Jasper van der Waa Hans Stokking BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation Frontiers in Human Neuroscience EEG brain computer interface neuroadaptive technology virtual reality head mounted display head rotation |
| title | BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation |
| title_full | BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation |
| title_fullStr | BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation |
| title_full_unstemmed | BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation |
| title_short | BCI to Potentially Enhance Streaming Images to a VR Headset by Predicting Head Rotation |
| title_sort | bci to potentially enhance streaming images to a vr headset by predicting head rotation |
| topic | EEG brain computer interface neuroadaptive technology virtual reality head mounted display head rotation |
| url | https://www.frontiersin.org/article/10.3389/fnhum.2018.00420/full |
| work_keys_str_mv | AT annemariebrouwer bcitopotentiallyenhancestreamingimagestoavrheadsetbypredictingheadrotation AT jaspervanderwaa bcitopotentiallyenhancestreamingimagestoavrheadsetbypredictingheadrotation AT hansstokking bcitopotentiallyenhancestreamingimagestoavrheadsetbypredictingheadrotation |