Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size.
The p value obtained from a significance test provides no information about the magnitude or importance of the underlying phenomenon. Therefore, additional reporting of effect size is often recommended. Effect sizes are theoretically independent from sample size. Yet this may not hold true empirical...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2014-01-01
|
| Series: | PLoS ONE |
| Online Access: | http://europepmc.org/articles/PMC4156299?pdf=render |
| _version_ | 1818948998620250112 |
|---|---|
| author | Anton Kühberger Astrid Fritz Thomas Scherndl |
| author_facet | Anton Kühberger Astrid Fritz Thomas Scherndl |
| author_sort | Anton Kühberger |
| collection | DOAJ |
| description | The p value obtained from a significance test provides no information about the magnitude or importance of the underlying phenomenon. Therefore, additional reporting of effect size is often recommended. Effect sizes are theoretically independent from sample size. Yet this may not hold true empirically: non-independence could indicate publication bias.We investigate whether effect size is independent from sample size in psychological research. We randomly sampled 1,000 psychological articles from all areas of psychological research. We extracted p values, effect sizes, and sample sizes of all empirical papers, and calculated the correlation between effect size and sample size, and investigated the distribution of p values.We found a negative correlation of r = -.45 [95% CI: -.53; -.35] between effect size and sample size. In addition, we found an inordinately high number of p values just passing the boundary of significance. Additional data showed that neither implicit nor explicit power analysis could account for this pattern of findings.The negative correlation between effect size and samples size, and the biased distribution of p values indicate pervasive publication bias in the entire field of psychology. |
| first_indexed | 2024-12-20T08:55:42Z |
| format | Article |
| id | doaj.art-573d1930cd5c4dc1a267fd8d7b5a32ec |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-20T08:55:42Z |
| publishDate | 2014-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-573d1930cd5c4dc1a267fd8d7b5a32ec2022-12-21T19:46:01ZengPublic Library of Science (PLoS)PLoS ONE1932-62032014-01-0199e10582510.1371/journal.pone.0105825Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size.Anton KühbergerAstrid FritzThomas ScherndlThe p value obtained from a significance test provides no information about the magnitude or importance of the underlying phenomenon. Therefore, additional reporting of effect size is often recommended. Effect sizes are theoretically independent from sample size. Yet this may not hold true empirically: non-independence could indicate publication bias.We investigate whether effect size is independent from sample size in psychological research. We randomly sampled 1,000 psychological articles from all areas of psychological research. We extracted p values, effect sizes, and sample sizes of all empirical papers, and calculated the correlation between effect size and sample size, and investigated the distribution of p values.We found a negative correlation of r = -.45 [95% CI: -.53; -.35] between effect size and sample size. In addition, we found an inordinately high number of p values just passing the boundary of significance. Additional data showed that neither implicit nor explicit power analysis could account for this pattern of findings.The negative correlation between effect size and samples size, and the biased distribution of p values indicate pervasive publication bias in the entire field of psychology.http://europepmc.org/articles/PMC4156299?pdf=render |
| spellingShingle | Anton Kühberger Astrid Fritz Thomas Scherndl Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. PLoS ONE |
| title | Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. |
| title_full | Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. |
| title_fullStr | Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. |
| title_full_unstemmed | Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. |
| title_short | Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. |
| title_sort | publication bias in psychology a diagnosis based on the correlation between effect size and sample size |
| url | http://europepmc.org/articles/PMC4156299?pdf=render |
| work_keys_str_mv | AT antonkuhberger publicationbiasinpsychologyadiagnosisbasedonthecorrelationbetweeneffectsizeandsamplesize AT astridfritz publicationbiasinpsychologyadiagnosisbasedonthecorrelationbetweeneffectsizeandsamplesize AT thomasscherndl publicationbiasinpsychologyadiagnosisbasedonthecorrelationbetweeneffectsizeandsamplesize |