Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing
Copyright © 2020 The Authors. Cognitive Science published by Wiley Periodicals, Inc. on behalf of Cognitive Science Society (CSS) A key component of research on human sentence processing is to characterize the processing difficulty associated with the comprehension of words in context. Models that e...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135946 |
| _version_ | 1811082348686475264 |
|---|---|
| author | Futrell, Richard Gibson, Edward Levy, Roger P |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Futrell, Richard Gibson, Edward Levy, Roger P |
| author_sort | Futrell, Richard |
| collection | MIT |
| description | Copyright © 2020 The Authors. Cognitive Science published by Wiley Periodicals, Inc. on behalf of Cognitive Science Society (CSS) A key component of research on human sentence processing is to characterize the processing difficulty associated with the comprehension of words in context. Models that explain and predict this difficulty can be broadly divided into two kinds, expectation-based and memory-based. In this work, we present a new model of incremental sentence processing difficulty that unifies and extends key features of both kinds of models. Our model, lossy-context surprisal, holds that the processing difficulty at a word in context is proportional to the surprisal of the word given a lossy memory representation of the context—that is, a memory representation that does not contain complete information about previous words. We show that this model provides an intuitive explanation for an outstanding puzzle involving interactions of memory and expectations: language-dependent structural forgetting, where the effects of memory on sentence processing appear to be moderated by language statistics. Furthermore, we demonstrate that dependency locality effects, a signature prediction of memory-based theories, can be derived from lossy-context surprisal as a special case of a novel, more general principle called information locality. |
| first_indexed | 2024-09-23T12:01:42Z |
| format | Article |
| id | mit-1721.1/135946 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T12:01:42Z |
| publishDate | 2021 |
| publisher | Wiley |
| record_format | dspace |
| spelling | mit-1721.1/1359462024-01-02T18:54:33Z Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing Futrell, Richard Gibson, Edward Levy, Roger P Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Copyright © 2020 The Authors. Cognitive Science published by Wiley Periodicals, Inc. on behalf of Cognitive Science Society (CSS) A key component of research on human sentence processing is to characterize the processing difficulty associated with the comprehension of words in context. Models that explain and predict this difficulty can be broadly divided into two kinds, expectation-based and memory-based. In this work, we present a new model of incremental sentence processing difficulty that unifies and extends key features of both kinds of models. Our model, lossy-context surprisal, holds that the processing difficulty at a word in context is proportional to the surprisal of the word given a lossy memory representation of the context—that is, a memory representation that does not contain complete information about previous words. We show that this model provides an intuitive explanation for an outstanding puzzle involving interactions of memory and expectations: language-dependent structural forgetting, where the effects of memory on sentence processing appear to be moderated by language statistics. Furthermore, we demonstrate that dependency locality effects, a signature prediction of memory-based theories, can be derived from lossy-context surprisal as a special case of a novel, more general principle called information locality. 2021-10-27T20:30:05Z 2021-10-27T20:30:05Z 2020 2021-03-23T17:10:14Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135946 en 10.1111/COGS.12814 Cognitive Science Creative Commons Attribution NonCommercial License 4.0 https://creativecommons.org/licenses/by-nc/4.0/ application/pdf Wiley Wiley |
| spellingShingle | Futrell, Richard Gibson, Edward Levy, Roger P Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing |
| title | Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing |
| title_full | Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing |
| title_fullStr | Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing |
| title_full_unstemmed | Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing |
| title_short | Lossy‐Context Surprisal: An Information‐Theoretic Model of Memory Effects in Sentence Processing |
| title_sort | lossy context surprisal an information theoretic model of memory effects in sentence processing |
| url | https://hdl.handle.net/1721.1/135946 |
| work_keys_str_mv | AT futrellrichard lossycontextsurprisalaninformationtheoreticmodelofmemoryeffectsinsentenceprocessing AT gibsonedward lossycontextsurprisalaninformationtheoreticmodelofmemoryeffectsinsentenceprocessing AT levyrogerp lossycontextsurprisalaninformationtheoreticmodelofmemoryeffectsinsentenceprocessing |