BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems
The Brain-Computer Interface (BCI) was envisioned as an assistive technology option for people with severe movement impairments. The traditional synchronous event-related potential (ERP) BCI design uses a fixed communication speed and is vulnerable to variations in attention. Recent ERP BCI designs...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10271742/ |
| _version_ | 1827792869143871488 |
|---|---|
| author | Guoxuan Ma Jian Kang David E. Thompson Jane E. Huggins |
| author_facet | Guoxuan Ma Jian Kang David E. Thompson Jane E. Huggins |
| author_sort | Guoxuan Ma |
| collection | DOAJ |
| description | The Brain-Computer Interface (BCI) was envisioned as an assistive technology option for people with severe movement impairments. The traditional synchronous event-related potential (ERP) BCI design uses a fixed communication speed and is vulnerable to variations in attention. Recent ERP BCI designs have added asynchronous features, including abstention and dynamic stopping, but it remains a open question of how to evaluate asynchronous BCI performance. In this work, we build on the BCI-Utility metric to create the first evaluation metric with special consideration of the asynchronous features of self-paced BCIs. This metric considers accuracy as all of the following three – probability of a correct selection when a selection was intended, probability of making a selection when a selection was intended, and probability of an abstention when an abstention was intended. Further, it considers the average time required for a selection when using dynamic stopping and the proportion of intended selections versus abstentions. We establish the validity of the derived metric via extensive simulations, and illustrate and discuss its practical usage on real-world BCI data. We describe the relative contribution of different inputs with plots of BCI-Utility curves under different parameter settings. Generally, the BCI-Utility metric increases as any of the accuracy values increase and decreases as the expected time for an intended selection increases. Furthermore, in many situations, we find shortening the expected time of an intended selection is the most effective way to improve the BCI-Utility, which necessitates the advancement of asynchronous BCI systems capable of accurate abstention and dynamic stopping. |
| first_indexed | 2024-03-11T18:09:22Z |
| format | Article |
| id | doaj.art-8726cecd0e654fd996d7602e0e48f820 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-11T18:09:22Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-8726cecd0e654fd996d7602e0e48f8202023-10-16T23:00:11ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01313968397710.1109/TNSRE.2023.332212510271742BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface SystemsGuoxuan Ma0https://orcid.org/0009-0003-2479-7800Jian Kang1David E. Thompson2https://orcid.org/0000-0002-1897-2743Jane E. Huggins3https://orcid.org/0000-0001-8709-4350Department of Biostatistics, University of Michigan, Ann Arbor, MI, USADepartment of Biostatistics, University of Michigan, Ann Arbor, MI, USADepartment of Electrical and Computer Engineering, Kansas State University, Manhattan, KS, USAPhysical Medicine and Rehabilitation Department, Michigan Medicine, Department of Biomedical Engineering, and the College of Engineering, University of Michigan, Ann Arbor, MI, USAThe Brain-Computer Interface (BCI) was envisioned as an assistive technology option for people with severe movement impairments. The traditional synchronous event-related potential (ERP) BCI design uses a fixed communication speed and is vulnerable to variations in attention. Recent ERP BCI designs have added asynchronous features, including abstention and dynamic stopping, but it remains a open question of how to evaluate asynchronous BCI performance. In this work, we build on the BCI-Utility metric to create the first evaluation metric with special consideration of the asynchronous features of self-paced BCIs. This metric considers accuracy as all of the following three – probability of a correct selection when a selection was intended, probability of making a selection when a selection was intended, and probability of an abstention when an abstention was intended. Further, it considers the average time required for a selection when using dynamic stopping and the proportion of intended selections versus abstentions. We establish the validity of the derived metric via extensive simulations, and illustrate and discuss its practical usage on real-world BCI data. We describe the relative contribution of different inputs with plots of BCI-Utility curves under different parameter settings. Generally, the BCI-Utility metric increases as any of the accuracy values increase and decreases as the expected time for an intended selection increases. Furthermore, in many situations, we find shortening the expected time of an intended selection is the most effective way to improve the BCI-Utility, which necessitates the advancement of asynchronous BCI systems capable of accurate abstention and dynamic stopping.https://ieeexplore.ieee.org/document/10271742/Brain–computer interface (BCI)BCI performance metricsERP BCI speller |
| spellingShingle | Guoxuan Ma Jian Kang David E. Thompson Jane E. Huggins BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems IEEE Transactions on Neural Systems and Rehabilitation Engineering Brain–computer interface (BCI) BCI performance metrics ERP BCI speller |
| title | BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems |
| title_full | BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems |
| title_fullStr | BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems |
| title_full_unstemmed | BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems |
| title_short | BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems |
| title_sort | bci utility metric for asynchronous p300 brain computer interface systems |
| topic | Brain–computer interface (BCI) BCI performance metrics ERP BCI speller |
| url | https://ieeexplore.ieee.org/document/10271742/ |
| work_keys_str_mv | AT guoxuanma bciutilitymetricforasynchronousp300braincomputerinterfacesystems AT jiankang bciutilitymetricforasynchronousp300braincomputerinterfacesystems AT davidethompson bciutilitymetricforasynchronousp300braincomputerinterfacesystems AT janeehuggins bciutilitymetricforasynchronousp300braincomputerinterfacesystems |