Power Analysis and Effect Size in Mixed Effects Models: A Tutorial
In psychology, attempts to replicate published findings are less successful than expected. For properly powered studies replication rate should be around 80%, whereas in practice less than 40% of the studies selected from different areas of psychology can be replicated. Researchers in cognitive psyc...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Ubiquity Press
2018-01-01
|
| Series: | Journal of Cognition |
| Subjects: | |
| Online Access: | https://www.journalofcognition.org/articles/10 |
| _version_ | 1811221824593199104 |
|---|---|
| author | Marc Brysbaert Michaël Stevens |
| author_facet | Marc Brysbaert Michaël Stevens |
| author_sort | Marc Brysbaert |
| collection | DOAJ |
| description | In psychology, attempts to replicate published findings are less successful than expected. For properly powered studies replication rate should be around 80%, whereas in practice less than 40% of the studies selected from different areas of psychology can be replicated. Researchers in cognitive psychology are hindered in estimating the power of their studies, because the designs they use present a sample of stimulus materials to a sample of participants, a situation not covered by most power formulas. To remedy the situation, we review the literature related to the topic and introduce recent software packages, which we apply to the data of two masked priming studies with high power. We checked how we could estimate the power of each study and how much they could be reduced to remain powerful enough. On the basis of this analysis, we recommend that a properly powered reaction time experiment with repeated measures has at least 1,600 word observations per condition (e.g., 40 participants, 40 stimuli). This is considerably more than current practice. We also show that researchers must include the number of observations in meta-analyses because the effect sizes currently reported depend on the number of stimuli presented to the participants. Our analyses can easily be applied to new datasets gathered. |
| first_indexed | 2024-04-12T08:06:46Z |
| format | Article |
| id | doaj.art-86f7a356eea546a5bda646ed04e9615b |
| institution | Directory Open Access Journal |
| issn | 2514-4820 |
| language | English |
| last_indexed | 2024-04-12T08:06:46Z |
| publishDate | 2018-01-01 |
| publisher | Ubiquity Press |
| record_format | Article |
| series | Journal of Cognition |
| spelling | doaj.art-86f7a356eea546a5bda646ed04e9615b2022-12-22T03:41:08ZengUbiquity PressJournal of Cognition2514-48202018-01-011110.5334/joc.109Power Analysis and Effect Size in Mixed Effects Models: A TutorialMarc Brysbaert0Michaël Stevens1Ghent UniversityDepartment of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 GentIn psychology, attempts to replicate published findings are less successful than expected. For properly powered studies replication rate should be around 80%, whereas in practice less than 40% of the studies selected from different areas of psychology can be replicated. Researchers in cognitive psychology are hindered in estimating the power of their studies, because the designs they use present a sample of stimulus materials to a sample of participants, a situation not covered by most power formulas. To remedy the situation, we review the literature related to the topic and introduce recent software packages, which we apply to the data of two masked priming studies with high power. We checked how we could estimate the power of each study and how much they could be reduced to remain powerful enough. On the basis of this analysis, we recommend that a properly powered reaction time experiment with repeated measures has at least 1,600 word observations per condition (e.g., 40 participants, 40 stimuli). This is considerably more than current practice. We also show that researchers must include the number of observations in meta-analyses because the effect sizes currently reported depend on the number of stimuli presented to the participants. Our analyses can easily be applied to new datasets gathered.https://www.journalofcognition.org/articles/10power analysiseffect sizemixed effects modelsrandom factorsF1 analysisF2 analysis |
| spellingShingle | Marc Brysbaert Michaël Stevens Power Analysis and Effect Size in Mixed Effects Models: A Tutorial Journal of Cognition power analysis effect size mixed effects models random factors F1 analysis F2 analysis |
| title | Power Analysis and Effect Size in Mixed Effects Models: A Tutorial |
| title_full | Power Analysis and Effect Size in Mixed Effects Models: A Tutorial |
| title_fullStr | Power Analysis and Effect Size in Mixed Effects Models: A Tutorial |
| title_full_unstemmed | Power Analysis and Effect Size in Mixed Effects Models: A Tutorial |
| title_short | Power Analysis and Effect Size in Mixed Effects Models: A Tutorial |
| title_sort | power analysis and effect size in mixed effects models a tutorial |
| topic | power analysis effect size mixed effects models random factors F1 analysis F2 analysis |
| url | https://www.journalofcognition.org/articles/10 |
| work_keys_str_mv | AT marcbrysbaert poweranalysisandeffectsizeinmixedeffectsmodelsatutorial AT michaelstevens poweranalysisandeffectsizeinmixedeffectsmodelsatutorial |