Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface
The article presents an overview of several studies in the field of Brain–Computer Interfaces (BCIs), the requirements for the architecture of such promising devices, as well as multi-modal BCI for drone control in a smart-city environment. Distinctive features of the proposed solution are the simpl...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Engineering Proceedings |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2673-4591/33/1/43 |
| _version_ | 1797381195924766720 |
|---|---|
| author | Daniyar Wolf Mark Mamchenko Elena Jharko |
| author_facet | Daniyar Wolf Mark Mamchenko Elena Jharko |
| author_sort | Daniyar Wolf |
| collection | DOAJ |
| description | The article presents an overview of several studies in the field of Brain–Computer Interfaces (BCIs), the requirements for the architecture of such promising devices, as well as multi-modal BCI for drone control in a smart-city environment. Distinctive features of the proposed solution are the simplicity of the architecture (the use of only one smartphone for both receiving and processing bio-signals from the headset and transmitting commands to the drone), an open-source software solution for signal processing, generating, and sending commands to the unmanned aerial vehicle (UAV), as well as multimodality of the BCI (the use of both electroencephalographic (EEG) and electrooculographic (EOG) signals of the operator). For bio-signal acquisition, we used the NeuroSky Mindwave Mobile 2 headset, which is connected to an Android-based smartphone via Bluetooth. The developed Android application (Tello NeuroSky) processes signals from the headset and generates and transmits commands to the DJI Tello UAV via Wi-Fi. The decrease (depression) and increase of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>α</mi></semantics></math></inline-formula>- and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>β</mi></semantics></math></inline-formula>-rhythms of the brain, as well as EOG signals that occur during blinking were the triggers for UAV commands. The developed software allows the manual setting of the minimum, maximum and threshold values for the processed bio-signals. The following commands for the UAV were implemented: take-off, landing, forward movement, and backwards movement. Two threads of the smartphone’s central processing unit (CPU) were utilized when processing signals in the software to increase the performance: for signal processing (1-D Daubechies 2 (db2) wavelet transform) and updating data on the diagrams, and for generating and transmitting commands to the drone. |
| first_indexed | 2024-03-08T20:47:53Z |
| format | Article |
| id | doaj.art-8c4075b74d5a44999e7ccda68cff9001 |
| institution | Directory Open Access Journal |
| issn | 2673-4591 |
| language | English |
| last_indexed | 2024-03-08T20:47:53Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Engineering Proceedings |
| spelling | doaj.art-8c4075b74d5a44999e7ccda68cff90012023-12-22T14:06:49ZengMDPI AGEngineering Proceedings2673-45912023-06-013314310.3390/engproc2023033043Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer InterfaceDaniyar Wolf0Mark Mamchenko1Elena Jharko2V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Profsoyuznaya Street 65, Moscow 117342, RussiaV.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Profsoyuznaya Street 65, Moscow 117342, RussiaV.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Profsoyuznaya Street 65, Moscow 117342, RussiaThe article presents an overview of several studies in the field of Brain–Computer Interfaces (BCIs), the requirements for the architecture of such promising devices, as well as multi-modal BCI for drone control in a smart-city environment. Distinctive features of the proposed solution are the simplicity of the architecture (the use of only one smartphone for both receiving and processing bio-signals from the headset and transmitting commands to the drone), an open-source software solution for signal processing, generating, and sending commands to the unmanned aerial vehicle (UAV), as well as multimodality of the BCI (the use of both electroencephalographic (EEG) and electrooculographic (EOG) signals of the operator). For bio-signal acquisition, we used the NeuroSky Mindwave Mobile 2 headset, which is connected to an Android-based smartphone via Bluetooth. The developed Android application (Tello NeuroSky) processes signals from the headset and generates and transmits commands to the DJI Tello UAV via Wi-Fi. The decrease (depression) and increase of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>α</mi></semantics></math></inline-formula>- and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>β</mi></semantics></math></inline-formula>-rhythms of the brain, as well as EOG signals that occur during blinking were the triggers for UAV commands. The developed software allows the manual setting of the minimum, maximum and threshold values for the processed bio-signals. The following commands for the UAV were implemented: take-off, landing, forward movement, and backwards movement. Two threads of the smartphone’s central processing unit (CPU) were utilized when processing signals in the software to increase the performance: for signal processing (1-D Daubechies 2 (db2) wavelet transform) and updating data on the diagrams, and for generating and transmitting commands to the drone.https://www.mdpi.com/2673-4591/33/1/43Brain–Computer Interface (BCI)Brain–Machine Interface (BMI)unmanned vehicleunmanned aerial vehicle (UAV)smart city |
| spellingShingle | Daniyar Wolf Mark Mamchenko Elena Jharko Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface Engineering Proceedings Brain–Computer Interface (BCI) Brain–Machine Interface (BMI) unmanned vehicle unmanned aerial vehicle (UAV) smart city |
| title | Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface |
| title_full | Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface |
| title_fullStr | Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface |
| title_full_unstemmed | Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface |
| title_short | Control of Unmanned Vehicles in Smart Cities Using a Multi-Modal Brain–Computer Interface |
| title_sort | control of unmanned vehicles in smart cities using a multi modal brain computer interface |
| topic | Brain–Computer Interface (BCI) Brain–Machine Interface (BMI) unmanned vehicle unmanned aerial vehicle (UAV) smart city |
| url | https://www.mdpi.com/2673-4591/33/1/43 |
| work_keys_str_mv | AT daniyarwolf controlofunmannedvehiclesinsmartcitiesusingamultimodalbraincomputerinterface AT markmamchenko controlofunmannedvehiclesinsmartcitiesusingamultimodalbraincomputerinterface AT elenajharko controlofunmannedvehiclesinsmartcitiesusingamultimodalbraincomputerinterface |