Decoding dynamic affective responses to naturalistic videos with shared neural patterns
This study explored the feasibility of using shared neural patterns from brief affective episodes (viewing affective pictures) to decode extended, dynamic affective sequences in a naturalistic experience (watching movie-trailers). Twenty-eight participants viewed pictures from the International Affe...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2020-08-01
|
| Series: | NeuroImage |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1053811920301051 |
| _version_ | 1819298432878116864 |
|---|---|
| author | Hang-Yee Chan Ale Smidts Vincent C. Schoots Alan G. Sanfey Maarten A.S. Boksem |
| author_facet | Hang-Yee Chan Ale Smidts Vincent C. Schoots Alan G. Sanfey Maarten A.S. Boksem |
| author_sort | Hang-Yee Chan |
| collection | DOAJ |
| description | This study explored the feasibility of using shared neural patterns from brief affective episodes (viewing affective pictures) to decode extended, dynamic affective sequences in a naturalistic experience (watching movie-trailers). Twenty-eight participants viewed pictures from the International Affective Picture System (IAPS) and, in a separate session, watched various movie-trailers. We first located voxels at bilateral occipital cortex (LOC) responsive to affective picture categories by GLM analysis, then performed between-subject hyperalignment on the LOC voxels based on their responses during movie-trailer watching. After hyperalignment, we trained between-subject machine learning classifiers on the affective pictures, and used the classifiers to decode affective states of an out-of-sample participant both during picture viewing and during movie-trailer watching. Within participants, neural classifiers identified valence and arousal categories of pictures, and tracked self-reported valence and arousal during video watching. In aggregate, neural classifiers produced valence and arousal time series that tracked the dynamic ratings of the movie-trailers obtained from a separate sample. Our findings provide further support for the possibility of using pre-trained neural representations to decode dynamic affective responses during a naturalistic experience. |
| first_indexed | 2024-12-24T05:29:49Z |
| format | Article |
| id | doaj.art-93db5d47a9e74cd88b6086e84d38991a |
| institution | Directory Open Access Journal |
| issn | 1095-9572 |
| language | English |
| last_indexed | 2024-12-24T05:29:49Z |
| publishDate | 2020-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | NeuroImage |
| spelling | doaj.art-93db5d47a9e74cd88b6086e84d38991a2022-12-21T17:13:12ZengElsevierNeuroImage1095-95722020-08-01216116618Decoding dynamic affective responses to naturalistic videos with shared neural patternsHang-Yee Chan0Ale Smidts1Vincent C. Schoots2Alan G. Sanfey3Maarten A.S. Boksem4Department of Marketing Management, Rotterdam School of Management, Erasmus University Rotterdam, the Netherlands; Corresponding author. Department of Marketing Management, Rotterdam School of Management, Burgemeester Oudlaan 50, 3062 PA, Rotterdam, the Netherlands.Department of Marketing Management, Rotterdam School of Management, Erasmus University Rotterdam, the NetherlandsDepartment of Marketing Management, Rotterdam School of Management, Erasmus University Rotterdam, the NetherlandsCentre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, NetherlandsDepartment of Marketing Management, Rotterdam School of Management, Erasmus University Rotterdam, the NetherlandsThis study explored the feasibility of using shared neural patterns from brief affective episodes (viewing affective pictures) to decode extended, dynamic affective sequences in a naturalistic experience (watching movie-trailers). Twenty-eight participants viewed pictures from the International Affective Picture System (IAPS) and, in a separate session, watched various movie-trailers. We first located voxels at bilateral occipital cortex (LOC) responsive to affective picture categories by GLM analysis, then performed between-subject hyperalignment on the LOC voxels based on their responses during movie-trailer watching. After hyperalignment, we trained between-subject machine learning classifiers on the affective pictures, and used the classifiers to decode affective states of an out-of-sample participant both during picture viewing and during movie-trailer watching. Within participants, neural classifiers identified valence and arousal categories of pictures, and tracked self-reported valence and arousal during video watching. In aggregate, neural classifiers produced valence and arousal time series that tracked the dynamic ratings of the movie-trailers obtained from a separate sample. Our findings provide further support for the possibility of using pre-trained neural representations to decode dynamic affective responses during a naturalistic experience.http://www.sciencedirect.com/science/article/pii/S1053811920301051 |
| spellingShingle | Hang-Yee Chan Ale Smidts Vincent C. Schoots Alan G. Sanfey Maarten A.S. Boksem Decoding dynamic affective responses to naturalistic videos with shared neural patterns NeuroImage |
| title | Decoding dynamic affective responses to naturalistic videos with shared neural patterns |
| title_full | Decoding dynamic affective responses to naturalistic videos with shared neural patterns |
| title_fullStr | Decoding dynamic affective responses to naturalistic videos with shared neural patterns |
| title_full_unstemmed | Decoding dynamic affective responses to naturalistic videos with shared neural patterns |
| title_short | Decoding dynamic affective responses to naturalistic videos with shared neural patterns |
| title_sort | decoding dynamic affective responses to naturalistic videos with shared neural patterns |
| url | http://www.sciencedirect.com/science/article/pii/S1053811920301051 |
| work_keys_str_mv | AT hangyeechan decodingdynamicaffectiveresponsestonaturalisticvideoswithsharedneuralpatterns AT alesmidts decodingdynamicaffectiveresponsestonaturalisticvideoswithsharedneuralpatterns AT vincentcschoots decodingdynamicaffectiveresponsestonaturalisticvideoswithsharedneuralpatterns AT alangsanfey decodingdynamicaffectiveresponsestonaturalisticvideoswithsharedneuralpatterns AT maartenasboksem decodingdynamicaffectiveresponsestonaturalisticvideoswithsharedneuralpatterns |