A meta-analysis of syntactic priming in language production
We performed an exhaustive meta-analysis of 73 peer-reviewed journal articles on syntactic priming from the seminal Bock (1986) paper through 2013. Extracting the effect size for each experiment and condition, where the effect size is the log odds ratio of the frequency of the primed structure X to...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Elsevier
2018
|
| Online Access: | http://hdl.handle.net/1721.1/119405 https://orcid.org/0000-0002-9786-8716 https://orcid.org/0000-0002-2656-6139 https://orcid.org/0000-0002-5912-883X |
| _version_ | 1826197902455734272 |
|---|---|
| author | James, Ariel Mahowald, Kyle Adam Futrell, Richard Landy Jones Gibson, Edward A |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences James, Ariel Mahowald, Kyle Adam Futrell, Richard Landy Jones Gibson, Edward A |
| author_sort | James, Ariel |
| collection | MIT |
| description | We performed an exhaustive meta-analysis of 73 peer-reviewed journal articles on syntactic priming from the seminal Bock (1986) paper through 2013. Extracting the effect size for each experiment and condition, where the effect size is the log odds ratio of the frequency of the primed structure X to the frequency of the unprimed structure Y, we found a robust effect of syntactic priming with an average weighted odds ratio of 1.67 when there is no lexical overlap and 3.26 when there is. That is, a construction X which occurs 50% of the time in the absence of priming would occur 63% if primed without lexical repetition and 77% of the time if primed with lexical repetition. The syntactic priming effect is robust across several different construction types and languages, and we found strong effects of lexical overlap on the size of the priming effect as well as interactions between lexical repetition and temporal lag and between lexical repetition and whether the priming occurred within or across languages. We also analyzed the distribution of p-values across experiments in order to estimate the average statistical power of experiments in our sample and to assess publication bias. Analyzing a subset of experiments in which the primary result of interest is whether a particular structure showed a priming effect, we did not find evidence of major p-hacking and the studies appear to have acceptable statistical power: 82%. However, analyzing a subset of experiments that focus not just on whether syntactic priming exists but on how syntactic priming is moderated by other variables (such as repetition of words in prime and target, the location of the testing room, and the memory of the speaker), we found that such studies are, on average, underpowered with estimated average power of 53%. Using a subset of 45 papers from our sample for which we received raw data, we estimated subject and item variation and give recommendations for appropriate sample size for future syntactic priming studies. Keywords: Syntactic priming, Meta-analysis, Statistical power |
| first_indexed | 2024-09-23T10:55:23Z |
| format | Article |
| id | mit-1721.1/119405 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T10:55:23Z |
| publishDate | 2018 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | mit-1721.1/1194052022-10-01T00:00:27Z A meta-analysis of syntactic priming in language production James, Ariel Mahowald, Kyle Adam Futrell, Richard Landy Jones Gibson, Edward A Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Gibson, Edward Mahowald, Kyle Adam Futrell, Richard Landy Jones Gibson, Edward A We performed an exhaustive meta-analysis of 73 peer-reviewed journal articles on syntactic priming from the seminal Bock (1986) paper through 2013. Extracting the effect size for each experiment and condition, where the effect size is the log odds ratio of the frequency of the primed structure X to the frequency of the unprimed structure Y, we found a robust effect of syntactic priming with an average weighted odds ratio of 1.67 when there is no lexical overlap and 3.26 when there is. That is, a construction X which occurs 50% of the time in the absence of priming would occur 63% if primed without lexical repetition and 77% of the time if primed with lexical repetition. The syntactic priming effect is robust across several different construction types and languages, and we found strong effects of lexical overlap on the size of the priming effect as well as interactions between lexical repetition and temporal lag and between lexical repetition and whether the priming occurred within or across languages. We also analyzed the distribution of p-values across experiments in order to estimate the average statistical power of experiments in our sample and to assess publication bias. Analyzing a subset of experiments in which the primary result of interest is whether a particular structure showed a priming effect, we did not find evidence of major p-hacking and the studies appear to have acceptable statistical power: 82%. However, analyzing a subset of experiments that focus not just on whether syntactic priming exists but on how syntactic priming is moderated by other variables (such as repetition of words in prime and target, the location of the testing room, and the memory of the speaker), we found that such studies are, on average, underpowered with estimated average power of 53%. Using a subset of 45 papers from our sample for which we received raw data, we estimated subject and item variation and give recommendations for appropriate sample size for future syntactic priming studies. Keywords: Syntactic priming, Meta-analysis, Statistical power American Society for Engineering Education. National Defense Science and Engineering Graduate Fellowship 2018-12-04T15:25:45Z 2018-12-04T15:25:45Z 2016-05 2016-03 Article http://purl.org/eprint/type/JournalArticle 0749-596X http://hdl.handle.net/1721.1/119405 Mahowald, Kyle, Ariel James, Richard Futrell, and Edward Gibson. “A Meta-Analysis of Syntactic Priming in Language Production.” Journal of Memory and Language 91 (December 2016): 5–27. https://orcid.org/0000-0002-9786-8716 https://orcid.org/0000-0002-2656-6139 https://orcid.org/0000-0002-5912-883X en_US https://doi.org/10.1016/j.jml.2016.03.009 Journal of Memory and Language Creative Commons Attribution-NonCommercial-NoDerivs License http://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Elsevier Prof. Gibson via Courtney Crummett |
| spellingShingle | James, Ariel Mahowald, Kyle Adam Futrell, Richard Landy Jones Gibson, Edward A A meta-analysis of syntactic priming in language production |
| title | A meta-analysis of syntactic priming in language production |
| title_full | A meta-analysis of syntactic priming in language production |
| title_fullStr | A meta-analysis of syntactic priming in language production |
| title_full_unstemmed | A meta-analysis of syntactic priming in language production |
| title_short | A meta-analysis of syntactic priming in language production |
| title_sort | meta analysis of syntactic priming in language production |
| url | http://hdl.handle.net/1721.1/119405 https://orcid.org/0000-0002-9786-8716 https://orcid.org/0000-0002-2656-6139 https://orcid.org/0000-0002-5912-883X |
| work_keys_str_mv | AT jamesariel ametaanalysisofsyntacticpriminginlanguageproduction AT mahowaldkyleadam ametaanalysisofsyntacticpriminginlanguageproduction AT futrellrichardlandyjones ametaanalysisofsyntacticpriminginlanguageproduction AT gibsonedwarda ametaanalysisofsyntacticpriminginlanguageproduction AT jamesariel metaanalysisofsyntacticpriminginlanguageproduction AT mahowaldkyleadam metaanalysisofsyntacticpriminginlanguageproduction AT futrellrichardlandyjones metaanalysisofsyntacticpriminginlanguageproduction AT gibsonedwarda metaanalysisofsyntacticpriminginlanguageproduction |