A Novel Multimodal Approach for Hybrid Brain–Computer Interface
Brain-computer interface (BCI) technologies have been widely used in many areas. In particular, non-invasive technologies such as electroencephalography (EEG) or near-infrared spectroscopy (NIRS) have been used to detect motor imagery, disease, or mental state. It has been already shown in literatur...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9091871/ |
| _version_ | 1831676691674562560 |
|---|---|
| author | Zhe Sun Zihao Huang Feng Duan Yu Liu |
| author_facet | Zhe Sun Zihao Huang Feng Duan Yu Liu |
| author_sort | Zhe Sun |
| collection | DOAJ |
| description | Brain-computer interface (BCI) technologies have been widely used in many areas. In particular, non-invasive technologies such as electroencephalography (EEG) or near-infrared spectroscopy (NIRS) have been used to detect motor imagery, disease, or mental state. It has been already shown in literature that the hybrid of EEG and NIRS has better results than their respective individual signals. The fusion algorithm for EEG and NIRS sources is the key to implement them in real-life applications. In this research, we propose three fusion methods for the hybrid of the EEG and NIRS-based brain-computer interface system: linear fusion, tensor fusion, and p th-order polynomial fusion. Firstly, our results prove that the hybrid BCI system is more accurate, as expected. Secondly, the p th-order polynomial fusion has the best classification results out of the three methods, and also shows improvements compared with previous studies. For a motion imagery task and a mental arithmetic task, the best detection accuracy in previous papers were 74.20% and 88.1%, whereas our accuracy achieved was 77.53% and 90.19%. Furthermore, unlike complex artificial neural network methods, our proposed methods are not as computationally demanding. |
| first_indexed | 2024-12-20T04:27:40Z |
| format | Article |
| id | doaj.art-253abce9e9a54be398b32916865cc1cb |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T04:27:40Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-253abce9e9a54be398b32916865cc1cb2022-12-21T19:53:27ZengIEEEIEEE Access2169-35362020-01-018899098991810.1109/ACCESS.2020.29942269091871A Novel Multimodal Approach for Hybrid Brain–Computer InterfaceZhe Sun0https://orcid.org/0000-0002-6531-0769Zihao Huang1https://orcid.org/0000-0002-8111-9278Feng Duan2https://orcid.org/0000-0002-2179-2460Yu Liu3Computational Engineering Applications Unit, Head Office for Information Systems and Cybersecurity, RIKEN, Wako, JapanCollege of Artificial Intelligence, Nankai University, Tianjin, ChinaCollege of Artificial Intelligence, Nankai University, Tianjin, ChinaKey Laboratory of Exercise and Health Sciences of Ministry of Education, Shanghai University of Sport, Shanghai, ChinaBrain-computer interface (BCI) technologies have been widely used in many areas. In particular, non-invasive technologies such as electroencephalography (EEG) or near-infrared spectroscopy (NIRS) have been used to detect motor imagery, disease, or mental state. It has been already shown in literature that the hybrid of EEG and NIRS has better results than their respective individual signals. The fusion algorithm for EEG and NIRS sources is the key to implement them in real-life applications. In this research, we propose three fusion methods for the hybrid of the EEG and NIRS-based brain-computer interface system: linear fusion, tensor fusion, and p th-order polynomial fusion. Firstly, our results prove that the hybrid BCI system is more accurate, as expected. Secondly, the p th-order polynomial fusion has the best classification results out of the three methods, and also shows improvements compared with previous studies. For a motion imagery task and a mental arithmetic task, the best detection accuracy in previous papers were 74.20% and 88.1%, whereas our accuracy achieved was 77.53% and 90.19%. Furthermore, unlike complex artificial neural network methods, our proposed methods are not as computationally demanding.https://ieeexplore.ieee.org/document/9091871/Brain-computer interfaceelectroencephalographynear-infrared spectroscopymultimodal signalpolynomial fusion |
| spellingShingle | Zhe Sun Zihao Huang Feng Duan Yu Liu A Novel Multimodal Approach for Hybrid Brain–Computer Interface IEEE Access Brain-computer interface electroencephalography near-infrared spectroscopy multimodal signal polynomial fusion |
| title | A Novel Multimodal Approach for Hybrid Brain–Computer Interface |
| title_full | A Novel Multimodal Approach for Hybrid Brain–Computer Interface |
| title_fullStr | A Novel Multimodal Approach for Hybrid Brain–Computer Interface |
| title_full_unstemmed | A Novel Multimodal Approach for Hybrid Brain–Computer Interface |
| title_short | A Novel Multimodal Approach for Hybrid Brain–Computer Interface |
| title_sort | novel multimodal approach for hybrid brain x2013 computer interface |
| topic | Brain-computer interface electroencephalography near-infrared spectroscopy multimodal signal polynomial fusion |
| url | https://ieeexplore.ieee.org/document/9091871/ |
| work_keys_str_mv | AT zhesun anovelmultimodalapproachforhybridbrainx2013computerinterface AT zihaohuang anovelmultimodalapproachforhybridbrainx2013computerinterface AT fengduan anovelmultimodalapproachforhybridbrainx2013computerinterface AT yuliu anovelmultimodalapproachforhybridbrainx2013computerinterface AT zhesun novelmultimodalapproachforhybridbrainx2013computerinterface AT zihaohuang novelmultimodalapproachforhybridbrainx2013computerinterface AT fengduan novelmultimodalapproachforhybridbrainx2013computerinterface AT yuliu novelmultimodalapproachforhybridbrainx2013computerinterface |