Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices
Recently, visual encoding and decoding based on functional magnetic resonance imaging (fMRI) has had many achievements with the rapid development of deep network computation. In the human vision system, when people process the perceived visual content, visual information flows from primary visual co...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2019-07-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnins.2019.00692/full |
| _version_ | 1819056081818615808 |
|---|---|
| author | Kai Qiao Jian Chen Linyuan Wang Chi Zhang Lei Zeng Li Tong Bin Yan |
| author_facet | Kai Qiao Jian Chen Linyuan Wang Chi Zhang Lei Zeng Li Tong Bin Yan |
| author_sort | Kai Qiao |
| collection | DOAJ |
| description | Recently, visual encoding and decoding based on functional magnetic resonance imaging (fMRI) has had many achievements with the rapid development of deep network computation. In the human vision system, when people process the perceived visual content, visual information flows from primary visual cortices to high-level visual cortices and also vice versa based on the bottom-up and top-down manners, respectively. Inspired by the bidirectional information flows, we proposed a bidirectional recurrent neural network (BRNN)-based method to decode the corresponding categories from fMRI data. The forward and backward directions in the BRNN module characterized the bottom-up and top-down manners, respectively. The proposed method regarded the selected voxels in each visual area (V1, V2, V3, V4, and LO) as one node of the space sequence and fed it into the BRNN module, then combined the output of the BRNN module to decode categories with the subsequent fully connected softmax layer. This new method can use the hierarchical information representations and bidirectional information flows in human visual cortices more efficiently. Experiments demonstrated that our method could improve the accuracy of the three-level category decoding. Comparative analysis validated and revealed that correlative representations of categories were included in visual cortices because of the bidirectional information flows, in addition to the hierarchical, distributed, and complementary representations that accorded with previous studies. |
| first_indexed | 2024-12-21T13:17:45Z |
| format | Article |
| id | doaj.art-fc88a141342d40d499374ecdfdedc8eb |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-21T13:17:45Z |
| publishDate | 2019-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-fc88a141342d40d499374ecdfdedc8eb2022-12-21T19:02:41ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2019-07-011310.3389/fnins.2019.00692452607Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual CorticesKai QiaoJian ChenLinyuan WangChi ZhangLei ZengLi TongBin YanRecently, visual encoding and decoding based on functional magnetic resonance imaging (fMRI) has had many achievements with the rapid development of deep network computation. In the human vision system, when people process the perceived visual content, visual information flows from primary visual cortices to high-level visual cortices and also vice versa based on the bottom-up and top-down manners, respectively. Inspired by the bidirectional information flows, we proposed a bidirectional recurrent neural network (BRNN)-based method to decode the corresponding categories from fMRI data. The forward and backward directions in the BRNN module characterized the bottom-up and top-down manners, respectively. The proposed method regarded the selected voxels in each visual area (V1, V2, V3, V4, and LO) as one node of the space sequence and fed it into the BRNN module, then combined the output of the BRNN module to decode categories with the subsequent fully connected softmax layer. This new method can use the hierarchical information representations and bidirectional information flows in human visual cortices more efficiently. Experiments demonstrated that our method could improve the accuracy of the three-level category decoding. Comparative analysis validated and revealed that correlative representations of categories were included in visual cortices because of the bidirectional information flows, in addition to the hierarchical, distributed, and complementary representations that accorded with previous studies.https://www.frontiersin.org/article/10.3389/fnins.2019.00692/fullbrain decodingfunctional magnetic resonance imagingbidirectional recurrent neural networkbidirectional information flowsbottom-up mannertop-down manner |
| spellingShingle | Kai Qiao Jian Chen Linyuan Wang Chi Zhang Lei Zeng Li Tong Bin Yan Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices Frontiers in Neuroscience brain decoding functional magnetic resonance imaging bidirectional recurrent neural network bidirectional information flows bottom-up manner top-down manner |
| title | Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices |
| title_full | Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices |
| title_fullStr | Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices |
| title_full_unstemmed | Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices |
| title_short | Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices |
| title_sort | category decoding of visual stimuli from human brain activity using a bidirectional recurrent neural network to simulate bidirectional information flows in human visual cortices |
| topic | brain decoding functional magnetic resonance imaging bidirectional recurrent neural network bidirectional information flows bottom-up manner top-down manner |
| url | https://www.frontiersin.org/article/10.3389/fnins.2019.00692/full |
| work_keys_str_mv | AT kaiqiao categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices AT jianchen categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices AT linyuanwang categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices AT chizhang categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices AT leizeng categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices AT litong categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices AT binyan categorydecodingofvisualstimulifromhumanbrainactivityusingabidirectionalrecurrentneuralnetworktosimulatebidirectionalinformationflowsinhumanvisualcortices |