What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing
Abstract Visual narratives sometimes depict successive images with different characters in the same physical space; corpus analysis has revealed that this occurs more often in Japanese manga than American comics. We used event-related brain potentials to determine whether comprehension of “visual na...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2017-05-01
|
| Series: | Cognitive Research |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s41235-017-0064-5 |
| _version_ | 1818438386795413504 |
|---|---|
| author | Neil Cohn Marta Kutas |
| author_facet | Neil Cohn Marta Kutas |
| author_sort | Neil Cohn |
| collection | DOAJ |
| description | Abstract Visual narratives sometimes depict successive images with different characters in the same physical space; corpus analysis has revealed that this occurs more often in Japanese manga than American comics. We used event-related brain potentials to determine whether comprehension of “visual narrative conjunctions” invokes not only incremental mental updating as traditionally assumed, but also, as we propose, “grammatical” combinatoric processing. We thus crossed (non)/conjunction sequences with character (in)/congruity. Conjunctions elicited a larger anterior negativity (300–500 ms) than nonconjunctions, regardless of congruity, implicating “grammatical” processes. Conjunction and incongruity both elicited larger P600s (500–700 ms), indexing updating. Both conjunction effects were modulated by participants’ frequency of reading manga while growing up. Greater anterior negativity in frequent manga readers suggests more reliance on combinatoric processing; larger P600 effects in infrequent manga readers suggest more resources devoted to mental updating. As in language comprehension, it seems that processing conjunctions in visual narratives is not just mental updating but also partly grammatical, conditioned by comic readers’ experience with specific visual narrative structures. |
| first_indexed | 2024-12-14T17:39:45Z |
| format | Article |
| id | doaj.art-5c7b0c1d03254438b2874a65cf1c3a07 |
| institution | Directory Open Access Journal |
| issn | 2365-7464 |
| language | English |
| last_indexed | 2024-12-14T17:39:45Z |
| publishDate | 2017-05-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Cognitive Research |
| spelling | doaj.art-5c7b0c1d03254438b2874a65cf1c3a072022-12-21T22:52:53ZengSpringerOpenCognitive Research2365-74642017-05-012111310.1186/s41235-017-0064-5What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processingNeil Cohn0Marta Kutas1Center for Research in Language, University of California, San DiegoDepartment of Cognitive Science, University of California, San DiegoAbstract Visual narratives sometimes depict successive images with different characters in the same physical space; corpus analysis has revealed that this occurs more often in Japanese manga than American comics. We used event-related brain potentials to determine whether comprehension of “visual narrative conjunctions” invokes not only incremental mental updating as traditionally assumed, but also, as we propose, “grammatical” combinatoric processing. We thus crossed (non)/conjunction sequences with character (in)/congruity. Conjunctions elicited a larger anterior negativity (300–500 ms) than nonconjunctions, regardless of congruity, implicating “grammatical” processes. Conjunction and incongruity both elicited larger P600s (500–700 ms), indexing updating. Both conjunction effects were modulated by participants’ frequency of reading manga while growing up. Greater anterior negativity in frequent manga readers suggests more reliance on combinatoric processing; larger P600 effects in infrequent manga readers suggest more resources devoted to mental updating. As in language comprehension, it seems that processing conjunctions in visual narratives is not just mental updating but also partly grammatical, conditioned by comic readers’ experience with specific visual narrative structures.http://link.springer.com/article/10.1186/s41235-017-0064-5Visual languageVisual narrative grammarDiscourseComicsP600Left anterior negativity |
| spellingShingle | Neil Cohn Marta Kutas What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing Cognitive Research Visual language Visual narrative grammar Discourse Comics P600 Left anterior negativity |
| title | What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing |
| title_full | What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing |
| title_fullStr | What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing |
| title_full_unstemmed | What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing |
| title_short | What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing |
| title_sort | what s your neural function visual narrative conjunction grammar meaning and fluency in sequential image processing |
| topic | Visual language Visual narrative grammar Discourse Comics P600 Left anterior negativity |
| url | http://link.springer.com/article/10.1186/s41235-017-0064-5 |
| work_keys_str_mv | AT neilcohn whatsyourneuralfunctionvisualnarrativeconjunctiongrammarmeaningandfluencyinsequentialimageprocessing AT martakutas whatsyourneuralfunctionvisualnarrativeconjunctiongrammarmeaningandfluencyinsequentialimageprocessing |