Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand
Millions of people worldwide suffer motor or sensory impairment due to stroke, spinal cord injury, multiple sclerosis, traumatic brain injury, diabetes, and motor neuron diseases such as ALS (amyotrophic lateral sclerosis). A brain-computer interface (BCI), which links the brain directly to a comput...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-08-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnins.2021.699631/full |
| _version_ | 1818888883485540352 |
|---|---|
| author | Chad Bouton Chad Bouton Chad Bouton Nikunj Bhagat Nikunj Bhagat Santosh Chandrasekaran Santosh Chandrasekaran Jose Herrero Jose Herrero Jose Herrero Noah Markowitz Elizabeth Espinal Elizabeth Espinal Joo-won Kim Richard Ramdeo Richard Ramdeo Junqian Xu Matthew F. Glasser Stephan Bickel Stephan Bickel Stephan Bickel Stephan Bickel Stephan Bickel Ashesh Mehta Ashesh Mehta Ashesh Mehta Ashesh Mehta |
| author_facet | Chad Bouton Chad Bouton Chad Bouton Nikunj Bhagat Nikunj Bhagat Santosh Chandrasekaran Santosh Chandrasekaran Jose Herrero Jose Herrero Jose Herrero Noah Markowitz Elizabeth Espinal Elizabeth Espinal Joo-won Kim Richard Ramdeo Richard Ramdeo Junqian Xu Matthew F. Glasser Stephan Bickel Stephan Bickel Stephan Bickel Stephan Bickel Stephan Bickel Ashesh Mehta Ashesh Mehta Ashesh Mehta Ashesh Mehta |
| author_sort | Chad Bouton |
| collection | DOAJ |
| description | Millions of people worldwide suffer motor or sensory impairment due to stroke, spinal cord injury, multiple sclerosis, traumatic brain injury, diabetes, and motor neuron diseases such as ALS (amyotrophic lateral sclerosis). A brain-computer interface (BCI), which links the brain directly to a computer, offers a new way to study the brain and potentially restore impairments in patients living with these debilitating conditions. One of the challenges currently facing BCI technology, however, is to minimize surgical risk while maintaining efficacy. Minimally invasive techniques, such as stereoelectroencephalography (SEEG) have become more widely used in clinical applications in epilepsy patients since they can lead to fewer complications. SEEG depth electrodes also give access to sulcal and white matter areas of the brain but have not been widely studied in brain-computer interfaces. Here we show the first demonstration of decoding sulcal and subcortical activity related to both movement and tactile sensation in the human hand. Furthermore, we have compared decoding performance in SEEG-based depth recordings versus those obtained with electrocorticography electrodes (ECoG) placed on gyri. Initial poor decoding performance and the observation that most neural modulation patterns varied in amplitude trial-to-trial and were transient (significantly shorter than the sustained finger movements studied), led to the development of a feature selection method based on a repeatability metric using temporal correlation. An algorithm based on temporal correlation was developed to isolate features that consistently repeated (required for accurate decoding) and possessed information content related to movement or touch-related stimuli. We subsequently used these features, along with deep learning methods, to automatically classify various motor and sensory events for individual fingers with high accuracy. Repeating features were found in sulcal, gyral, and white matter areas and were predominantly phasic or phasic-tonic across a wide frequency range for both HD (high density) ECoG and SEEG recordings. These findings motivated the use of long short-term memory (LSTM) recurrent neural networks (RNNs) which are well-suited to handling transient input features. Combining temporal correlation-based feature selection with LSTM yielded decoding accuracies of up to 92.04 ± 1.51% for hand movements, up to 91.69 ± 0.49% for individual finger movements, and up to 83.49 ± 0.72% for focal tactile stimuli to individual finger pads while using a relatively small number of SEEG electrodes. These findings may lead to a new class of minimally invasive brain-computer interface systems in the future, increasing its applicability to a wide variety of conditions. |
| first_indexed | 2024-12-19T17:00:12Z |
| format | Article |
| id | doaj.art-33a478eaaf7a47098a2f2e775bd8796c |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-19T17:00:12Z |
| publishDate | 2021-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-33a478eaaf7a47098a2f2e775bd8796c2022-12-21T20:13:18ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2021-08-011510.3389/fnins.2021.699631699631Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human HandChad Bouton0Chad Bouton1Chad Bouton2Nikunj Bhagat3Nikunj Bhagat4Santosh Chandrasekaran5Santosh Chandrasekaran6Jose Herrero7Jose Herrero8Jose Herrero9Noah Markowitz10Elizabeth Espinal11Elizabeth Espinal12Joo-won Kim13Richard Ramdeo14Richard Ramdeo15Junqian Xu16Matthew F. Glasser17Stephan Bickel18Stephan Bickel19Stephan Bickel20Stephan Bickel21Stephan Bickel22Ashesh Mehta23Ashesh Mehta24Ashesh Mehta25Ashesh Mehta26Feinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesHofstra-Northwell Medical School, New York, NY, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesDepartment of Neurosurgery, Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesDepartment of Radiology and Psychiatry, Baylor College of Medicine, Houston, TX, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesDepartment of Radiology and Psychiatry, Baylor College of Medicine, Houston, TX, United StatesDepartment of Radiology and Neuroscience, Washington University in St. Louis, St. Louis, MO, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesHofstra-Northwell Medical School, New York, NY, United StatesDepartment of Neurosurgery, Northwell Health, New York, NY, United StatesDepartment of Neurology, Northwell Health, New York, NY, United StatesFeinstein Institutes for Medical Research at Northwell Health, New York, NY, United StatesInstitute of Bioelectronic Medicine, Feinstein Institutes for Medical Research, New York, NY, United StatesHofstra-Northwell Medical School, New York, NY, United StatesDepartment of Neurosurgery, Northwell Health, New York, NY, United StatesMillions of people worldwide suffer motor or sensory impairment due to stroke, spinal cord injury, multiple sclerosis, traumatic brain injury, diabetes, and motor neuron diseases such as ALS (amyotrophic lateral sclerosis). A brain-computer interface (BCI), which links the brain directly to a computer, offers a new way to study the brain and potentially restore impairments in patients living with these debilitating conditions. One of the challenges currently facing BCI technology, however, is to minimize surgical risk while maintaining efficacy. Minimally invasive techniques, such as stereoelectroencephalography (SEEG) have become more widely used in clinical applications in epilepsy patients since they can lead to fewer complications. SEEG depth electrodes also give access to sulcal and white matter areas of the brain but have not been widely studied in brain-computer interfaces. Here we show the first demonstration of decoding sulcal and subcortical activity related to both movement and tactile sensation in the human hand. Furthermore, we have compared decoding performance in SEEG-based depth recordings versus those obtained with electrocorticography electrodes (ECoG) placed on gyri. Initial poor decoding performance and the observation that most neural modulation patterns varied in amplitude trial-to-trial and were transient (significantly shorter than the sustained finger movements studied), led to the development of a feature selection method based on a repeatability metric using temporal correlation. An algorithm based on temporal correlation was developed to isolate features that consistently repeated (required for accurate decoding) and possessed information content related to movement or touch-related stimuli. We subsequently used these features, along with deep learning methods, to automatically classify various motor and sensory events for individual fingers with high accuracy. Repeating features were found in sulcal, gyral, and white matter areas and were predominantly phasic or phasic-tonic across a wide frequency range for both HD (high density) ECoG and SEEG recordings. These findings motivated the use of long short-term memory (LSTM) recurrent neural networks (RNNs) which are well-suited to handling transient input features. Combining temporal correlation-based feature selection with LSTM yielded decoding accuracies of up to 92.04 ± 1.51% for hand movements, up to 91.69 ± 0.49% for individual finger movements, and up to 83.49 ± 0.72% for focal tactile stimuli to individual finger pads while using a relatively small number of SEEG electrodes. These findings may lead to a new class of minimally invasive brain-computer interface systems in the future, increasing its applicability to a wide variety of conditions.https://www.frontiersin.org/articles/10.3389/fnins.2021.699631/fullneuroprostheticsstereoelectroencephalographysensorimotortactile stimulineural decoding |
| spellingShingle | Chad Bouton Chad Bouton Chad Bouton Nikunj Bhagat Nikunj Bhagat Santosh Chandrasekaran Santosh Chandrasekaran Jose Herrero Jose Herrero Jose Herrero Noah Markowitz Elizabeth Espinal Elizabeth Espinal Joo-won Kim Richard Ramdeo Richard Ramdeo Junqian Xu Matthew F. Glasser Stephan Bickel Stephan Bickel Stephan Bickel Stephan Bickel Stephan Bickel Ashesh Mehta Ashesh Mehta Ashesh Mehta Ashesh Mehta Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand Frontiers in Neuroscience neuroprosthetics stereoelectroencephalography sensorimotor tactile stimuli neural decoding |
| title | Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand |
| title_full | Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand |
| title_fullStr | Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand |
| title_full_unstemmed | Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand |
| title_short | Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand |
| title_sort | decoding neural activity in sulcal and white matter areas of the brain to accurately predict individual finger movement and tactile stimuli of the human hand |
| topic | neuroprosthetics stereoelectroencephalography sensorimotor tactile stimuli neural decoding |
| url | https://www.frontiersin.org/articles/10.3389/fnins.2021.699631/full |
| work_keys_str_mv | AT chadbouton decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT chadbouton decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT chadbouton decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT nikunjbhagat decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT nikunjbhagat decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT santoshchandrasekaran decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT santoshchandrasekaran decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT joseherrero decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT joseherrero decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT joseherrero decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT noahmarkowitz decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT elizabethespinal decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT elizabethespinal decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT joowonkim decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT richardramdeo decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT richardramdeo decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT junqianxu decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT matthewfglasser decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT stephanbickel decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT stephanbickel decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT stephanbickel decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT stephanbickel decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT stephanbickel decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT asheshmehta decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT asheshmehta decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT asheshmehta decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand AT asheshmehta decodingneuralactivityinsulcalandwhitematterareasofthebraintoaccuratelypredictindividualfingermovementandtactilestimuliofthehumanhand |