Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture
Electroencephalography (EEG) signals have a major impact on how well assistive rehabilitation devices work. These signals have become a common technique in recent studies to investigate human motion functions and behaviors. However, incorporating EEG signals to investigate motor planning or movement...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9916262/ |
| _version_ | 1811251197977296896 |
|---|---|
| author | Maged S. Al-Quraishi Irraivan Elamvazuthi Tong Boon Tang Muhammad S. Al-Qurishi Syed Hasan Adil Mansoor Ebrahim Alberto Borboni |
| author_facet | Maged S. Al-Quraishi Irraivan Elamvazuthi Tong Boon Tang Muhammad S. Al-Qurishi Syed Hasan Adil Mansoor Ebrahim Alberto Borboni |
| author_sort | Maged S. Al-Quraishi |
| collection | DOAJ |
| description | Electroencephalography (EEG) signals have a major impact on how well assistive rehabilitation devices work. These signals have become a common technique in recent studies to investigate human motion functions and behaviors. However, incorporating EEG signals to investigate motor planning or movement intention could benefit all patients who can plan motion but are unable to execute it. In this paper, the movement planning of the lower limb was investigated using EEG signal and bilateral movements were employed, including dorsiflexion and plantar flexion of the right and left ankle joint movements. The proposed system uses Continuous Wavelet Transform (CWT) to generate a time–frequency (TF) map of each EEG signal in the motor cortex and then uses the extracted images as input to a deep learning model for classification. Deep Learning (DL) models are created based on vision transformer architecture (ViT) which is the state-of-the-art of image classification and also the proposed models were compared with residual neural network (ResNet). The proposed technique reveals a significant classification performance for the multiclass problem (<inline-formula> <tex-math notation="LaTeX">$p < 0.0001$ </tex-math></inline-formula>) where the classification accuracy was <inline-formula> <tex-math notation="LaTeX">$97.33~\pm ~1.86$ </tex-math></inline-formula> % and the F score, recall and precision were <inline-formula> <tex-math notation="LaTeX">$97.32~\pm ~1.88$ </tex-math></inline-formula> %, <inline-formula> <tex-math notation="LaTeX">$97.30~\pm ~1.90$ </tex-math></inline-formula> % and <inline-formula> <tex-math notation="LaTeX">$97.36~\pm ~1.81$ </tex-math></inline-formula> % respectively. These results show that DL is a promising technique that can be applied to investigate the user’s movements intention from EEG signals and highlight the potential of the proposed model for the development of future brain-machine interface (BMI) for neurorehabilitation purposes. |
| first_indexed | 2024-04-12T16:16:02Z |
| format | Article |
| id | doaj.art-ff7308356712490aa73aafc24570c0b8 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T16:16:02Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-ff7308356712490aa73aafc24570c0b82022-12-22T03:25:43ZengIEEEIEEE Access2169-35362022-01-011010944610945910.1109/ACCESS.2022.32139969916262Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer ArchitectureMaged S. Al-Quraishi0https://orcid.org/0000-0003-0911-789XIrraivan Elamvazuthi1https://orcid.org/0000-0002-4721-9400Tong Boon Tang2https://orcid.org/0000-0002-5721-6828Muhammad S. Al-Qurishi3https://orcid.org/0000-0002-7594-7325Syed Hasan Adil4https://orcid.org/0000-0003-1280-6645Mansoor Ebrahim5https://orcid.org/0000-0002-2000-8398Alberto Borboni6Department of Electrical and Electronic Engineering, Smart Assistive and Rehabilitative Technology (SMART) Research Group, Universiti Teknologi PETRONAS, Bandar, Seri Iskandar, MalaysiaDepartment of Electrical and Electronic Engineering, Smart Assistive and Rehabilitative Technology (SMART) Research Group, Universiti Teknologi PETRONAS, Bandar, Seri Iskandar, MalaysiaDepartment of Electrical and Electronic Engineering, Smart Assistive and Rehabilitative Technology (SMART) Research Group, Universiti Teknologi PETRONAS, Bandar, Seri Iskandar, MalaysiaResearch Department, Research and Innovation Division, Elm Company, Riyadh, Saudi ArabiaFaculty of Engineering, Sciences and Technology, Iqra University, Karachi, PakistanFaculty of Engineering, Sciences and Technology, Iqra University, Karachi, PakistanMechanical and Industrial Engineering Department, Universita degli Studi di Brescia, Brescia, ItalyElectroencephalography (EEG) signals have a major impact on how well assistive rehabilitation devices work. These signals have become a common technique in recent studies to investigate human motion functions and behaviors. However, incorporating EEG signals to investigate motor planning or movement intention could benefit all patients who can plan motion but are unable to execute it. In this paper, the movement planning of the lower limb was investigated using EEG signal and bilateral movements were employed, including dorsiflexion and plantar flexion of the right and left ankle joint movements. The proposed system uses Continuous Wavelet Transform (CWT) to generate a time–frequency (TF) map of each EEG signal in the motor cortex and then uses the extracted images as input to a deep learning model for classification. Deep Learning (DL) models are created based on vision transformer architecture (ViT) which is the state-of-the-art of image classification and also the proposed models were compared with residual neural network (ResNet). The proposed technique reveals a significant classification performance for the multiclass problem (<inline-formula> <tex-math notation="LaTeX">$p < 0.0001$ </tex-math></inline-formula>) where the classification accuracy was <inline-formula> <tex-math notation="LaTeX">$97.33~\pm ~1.86$ </tex-math></inline-formula> % and the F score, recall and precision were <inline-formula> <tex-math notation="LaTeX">$97.32~\pm ~1.88$ </tex-math></inline-formula> %, <inline-formula> <tex-math notation="LaTeX">$97.30~\pm ~1.90$ </tex-math></inline-formula> % and <inline-formula> <tex-math notation="LaTeX">$97.36~\pm ~1.81$ </tex-math></inline-formula> % respectively. These results show that DL is a promising technique that can be applied to investigate the user’s movements intention from EEG signals and highlight the potential of the proposed model for the development of future brain-machine interface (BMI) for neurorehabilitation purposes.https://ieeexplore.ieee.org/document/9916262/Continuous wavelet transformdeep learningelectroencephalographymotor-related cortical potentialsvision transformers architecture |
| spellingShingle | Maged S. Al-Quraishi Irraivan Elamvazuthi Tong Boon Tang Muhammad S. Al-Qurishi Syed Hasan Adil Mansoor Ebrahim Alberto Borboni Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture IEEE Access Continuous wavelet transform deep learning electroencephalography motor-related cortical potentials vision transformers architecture |
| title | Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture |
| title_full | Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture |
| title_fullStr | Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture |
| title_full_unstemmed | Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture |
| title_short | Decoding the User’s Movements Preparation From EEG Signals Using Vision Transformer Architecture |
| title_sort | decoding the user x2019 s movements preparation from eeg signals using vision transformer architecture |
| topic | Continuous wavelet transform deep learning electroencephalography motor-related cortical potentials vision transformers architecture |
| url | https://ieeexplore.ieee.org/document/9916262/ |
| work_keys_str_mv | AT magedsalquraishi decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture AT irraivanelamvazuthi decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture AT tongboontang decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture AT muhammadsalqurishi decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture AT syedhasanadil decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture AT mansoorebrahim decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture AT albertoborboni decodingtheuserx2019smovementspreparationfromeegsignalsusingvisiontransformerarchitecture |