Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study
Both embodied and symbolic accounts of conceptual organization would predict partial sharing and partial differentiation between the neural activations seen for concepts activated via different stimulus modalities. But cross-participant and cross-session variability in BOLD activity patterns makes a...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2012-08-01
|
| Series: | Frontiers in Neuroinformatics |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fninf.2012.00024/full |
| _version_ | 1819260460377047040 |
|---|---|
| author | Hiroyuki eAkama Brian eMurphy Brian eMurphy Li eNa Yumiko eShimizu Massimo ePoesio Massimo ePoesio |
| author_facet | Hiroyuki eAkama Brian eMurphy Brian eMurphy Li eNa Yumiko eShimizu Massimo ePoesio Massimo ePoesio |
| author_sort | Hiroyuki eAkama |
| collection | DOAJ |
| description | Both embodied and symbolic accounts of conceptual organization would predict partial sharing and partial differentiation between the neural activations seen for concepts activated via different stimulus modalities. But cross-participant and cross-session variability in BOLD activity patterns makes analyses of such patterns with MVPA methods challenging. Here we examine the effect of cross-modal and individual variation on the machine learning analysis of fMRI data recorded during a word property generation task. We present the same set of living and non-living concepts (land-mammals, or work tools) to a cohort of Japanese participants in two sessions: the first using auditory presentation of spoken words; the second using visual presentation of words written in Japanese characters. Classification accuracies confirmed that these semantic categories could be detected in single trials, with within-session predictive accuracies of 80-90%. However cross-session prediction (learning from auditory-task data to classify data from the written-word-task, or vice-versa) suffered from a performance penalty, achieving 65-75% (still individually significant at p<<0.05). We carried out several follow-on analyses to investigate the reason for this shortfall, concluding that distributional differences in neither time nor space alone could account for it. Rather, combined spatio-temporal patterns of activity need to be identified for successful cross-session learning, and this suggests that feature selection strategies could be modified to take advantage of this. |
| first_indexed | 2024-12-23T19:26:15Z |
| format | Article |
| id | doaj.art-ad7951dd91dd4282802b361a4c4e6e80 |
| institution | Directory Open Access Journal |
| issn | 1662-5196 |
| language | English |
| last_indexed | 2024-12-23T19:26:15Z |
| publishDate | 2012-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroinformatics |
| spelling | doaj.art-ad7951dd91dd4282802b361a4c4e6e802022-12-21T17:34:02ZengFrontiers Media S.A.Frontiers in Neuroinformatics1662-51962012-08-01610.3389/fninf.2012.0002426363Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA StudyHiroyuki eAkama0Brian eMurphy1Brian eMurphy2Li eNa3Yumiko eShimizu4Massimo ePoesio5Massimo ePoesio6Tokyo Institute of TechnologyCarnegie Mellon UniversityUniversity of TrentoTokyo Institute of TechnologyTokyo City UniversityUniversity of TrentoUniversity of EssexBoth embodied and symbolic accounts of conceptual organization would predict partial sharing and partial differentiation between the neural activations seen for concepts activated via different stimulus modalities. But cross-participant and cross-session variability in BOLD activity patterns makes analyses of such patterns with MVPA methods challenging. Here we examine the effect of cross-modal and individual variation on the machine learning analysis of fMRI data recorded during a word property generation task. We present the same set of living and non-living concepts (land-mammals, or work tools) to a cohort of Japanese participants in two sessions: the first using auditory presentation of spoken words; the second using visual presentation of words written in Japanese characters. Classification accuracies confirmed that these semantic categories could be detected in single trials, with within-session predictive accuracies of 80-90%. However cross-session prediction (learning from auditory-task data to classify data from the written-word-task, or vice-versa) suffered from a performance penalty, achieving 65-75% (still individually significant at p<<0.05). We carried out several follow-on analyses to investigate the reason for this shortfall, concluding that distributional differences in neither time nor space alone could account for it. Rather, combined spatio-temporal patterns of activity need to be identified for successful cross-session learning, and this suggests that feature selection strategies could be modified to take advantage of this.http://journal.frontiersin.org/Journal/10.3389/fninf.2012.00024/fullfMRImachine learningembodimentMVPAcomputational neurolinguisticsGLM |
| spellingShingle | Hiroyuki eAkama Brian eMurphy Brian eMurphy Li eNa Yumiko eShimizu Massimo ePoesio Massimo ePoesio Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study Frontiers in Neuroinformatics fMRI machine learning embodiment MVPA computational neurolinguistics GLM |
| title | Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study |
| title_full | Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study |
| title_fullStr | Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study |
| title_full_unstemmed | Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study |
| title_short | Decoding Semantics across fMRI sessions with Different Stimulus Modalities: A practical MVPA Study |
| title_sort | decoding semantics across fmri sessions with different stimulus modalities a practical mvpa study |
| topic | fMRI machine learning embodiment MVPA computational neurolinguistics GLM |
| url | http://journal.frontiersin.org/Journal/10.3389/fninf.2012.00024/full |
| work_keys_str_mv | AT hiroyukieakama decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy AT brianemurphy decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy AT brianemurphy decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy AT liena decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy AT yumikoeshimizu decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy AT massimoepoesio decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy AT massimoepoesio decodingsemanticsacrossfmrisessionswithdifferentstimulusmodalitiesapracticalmvpastudy |