A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent
Laboratory demonstrations of brain-computer interface (BCI) systems show promise for reducing disability associated with paralysis by directly linking neural activity to the control of assistive devices. Surveys of potential users have revealed several key BCI performance criteria for clinical trans...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-10-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnins.2018.00763/full |
| _version_ | 1818370227801423872 |
|---|---|
| author | Nicholas D. Skomrock Michael A. Schwemmer Jordyn E. Ting Hemang R. Trivedi Gaurav Sharma Marcia A. Bockbrader Marcia A. Bockbrader David A. Friedenberg |
| author_facet | Nicholas D. Skomrock Michael A. Schwemmer Jordyn E. Ting Hemang R. Trivedi Gaurav Sharma Marcia A. Bockbrader Marcia A. Bockbrader David A. Friedenberg |
| author_sort | Nicholas D. Skomrock |
| collection | DOAJ |
| description | Laboratory demonstrations of brain-computer interface (BCI) systems show promise for reducing disability associated with paralysis by directly linking neural activity to the control of assistive devices. Surveys of potential users have revealed several key BCI performance criteria for clinical translation of such a system. Of these criteria, high accuracy, short response latencies, and multi-functionality are three key characteristics directly impacted by the neural decoding component of the BCI system, the algorithm that translates neural activity into control signals. Building a decoder that simultaneously addresses these three criteria is complicated because optimizing for one criterion may lead to undesirable changes in the other criteria. Unfortunately, there has been little work to date to quantify how decoder design simultaneously affects these performance characteristics. Here, we systematically explore the trade-off between accuracy, response latency, and multi-functionality for discrete movement classification using two different decoding strategies–a support vector machine (SVM) classifier which represents the current state-of-the-art for discrete movement classification in laboratory demonstrations and a proposed deep neural network (DNN) framework. We utilized historical intracortical recordings from a human tetraplegic study participant, who imagined performing several different hand and finger movements. For both decoders, we found that response time increases (i.e., slower reaction) and accuracy decreases as the number of functions increases. However, we also found that both the increase of response times and the decline in accuracy with additional functions is less for the DNN than the SVM. We also show that data preprocessing steps can affect the performance characteristics of the two decoders in drastically different ways. Finally, we evaluated the performance of our tetraplegic participant using the DNN decoder in real-time to control functional electrical stimulation (FES) of his paralyzed forearm. We compared his performance to that of able-bodied participants performing the same task, establishing a quantitative target for ideal BCI-FES performance on this task. Cumulatively, these results help quantify BCI decoder performance characteristics relevant to potential users and the complex interactions between them. |
| first_indexed | 2024-12-13T23:36:23Z |
| format | Article |
| id | doaj.art-c332b57ea07b45c1bc2307dedaf8fa6d |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-13T23:36:23Z |
| publishDate | 2018-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-c332b57ea07b45c1bc2307dedaf8fa6d2022-12-21T23:27:18ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2018-10-011210.3389/fnins.2018.00763387913A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement IntentNicholas D. Skomrock0Michael A. Schwemmer1Jordyn E. Ting2Hemang R. Trivedi3Gaurav Sharma4Marcia A. Bockbrader5Marcia A. Bockbrader6David A. Friedenberg7Advanced Analytics and Health Research, Battelle Memorial Institute, Columbus, OH, United StatesAdvanced Analytics and Health Research, Battelle Memorial Institute, Columbus, OH, United StatesMedical Devices and Neuromodulation, Battelle Memorial Institute, Columbus, OH, United StatesMedical Devices and Neuromodulation, Battelle Memorial Institute, Columbus, OH, United StatesMedical Devices and Neuromodulation, Battelle Memorial Institute, Columbus, OH, United StatesNeurological Institute, The Ohio State University, Columbus, OH, United StatesDepartment of Physical Medicine and Rehabilitation, The Ohio State University, Columbus, OH, United StatesAdvanced Analytics and Health Research, Battelle Memorial Institute, Columbus, OH, United StatesLaboratory demonstrations of brain-computer interface (BCI) systems show promise for reducing disability associated with paralysis by directly linking neural activity to the control of assistive devices. Surveys of potential users have revealed several key BCI performance criteria for clinical translation of such a system. Of these criteria, high accuracy, short response latencies, and multi-functionality are three key characteristics directly impacted by the neural decoding component of the BCI system, the algorithm that translates neural activity into control signals. Building a decoder that simultaneously addresses these three criteria is complicated because optimizing for one criterion may lead to undesirable changes in the other criteria. Unfortunately, there has been little work to date to quantify how decoder design simultaneously affects these performance characteristics. Here, we systematically explore the trade-off between accuracy, response latency, and multi-functionality for discrete movement classification using two different decoding strategies–a support vector machine (SVM) classifier which represents the current state-of-the-art for discrete movement classification in laboratory demonstrations and a proposed deep neural network (DNN) framework. We utilized historical intracortical recordings from a human tetraplegic study participant, who imagined performing several different hand and finger movements. For both decoders, we found that response time increases (i.e., slower reaction) and accuracy decreases as the number of functions increases. However, we also found that both the increase of response times and the decline in accuracy with additional functions is less for the DNN than the SVM. We also show that data preprocessing steps can affect the performance characteristics of the two decoders in drastically different ways. Finally, we evaluated the performance of our tetraplegic participant using the DNN decoder in real-time to control functional electrical stimulation (FES) of his paralyzed forearm. We compared his performance to that of able-bodied participants performing the same task, establishing a quantitative target for ideal BCI-FES performance on this task. Cumulatively, these results help quantify BCI decoder performance characteristics relevant to potential users and the complex interactions between them.https://www.frontiersin.org/article/10.3389/fnins.2018.00763/fullbrain-computer interfacedecodingmachine learningdeep learningsupport vector machinesresponse time |
| spellingShingle | Nicholas D. Skomrock Michael A. Schwemmer Jordyn E. Ting Hemang R. Trivedi Gaurav Sharma Marcia A. Bockbrader Marcia A. Bockbrader David A. Friedenberg A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent Frontiers in Neuroscience brain-computer interface decoding machine learning deep learning support vector machines response time |
| title | A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent |
| title_full | A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent |
| title_fullStr | A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent |
| title_full_unstemmed | A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent |
| title_short | A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent |
| title_sort | characterization of brain computer interface performance trade offs using support vector machines and deep neural networks to decode movement intent |
| topic | brain-computer interface decoding machine learning deep learning support vector machines response time |
| url | https://www.frontiersin.org/article/10.3389/fnins.2018.00763/full |
| work_keys_str_mv | AT nicholasdskomrock acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT michaelaschwemmer acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT jordyneting acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT hemangrtrivedi acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT gauravsharma acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT marciaabockbrader acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT marciaabockbrader acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT davidafriedenberg acharacterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT nicholasdskomrock characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT michaelaschwemmer characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT jordyneting characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT hemangrtrivedi characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT gauravsharma characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT marciaabockbrader characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT marciaabockbrader characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent AT davidafriedenberg characterizationofbraincomputerinterfaceperformancetradeoffsusingsupportvectormachinesanddeepneuralnetworkstodecodemovementintent |