Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values
The influential claim that most published results are false raised concerns about the trustworthiness and integrity of science. Since then, there have been numerous attempts to examine the rate of false-positive results that have failed to settle this question empirically. Here we propose a new way...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468063/?tool=EBI |
| _version_ | 1797692775216447488 |
|---|---|
| author | Ulrich Schimmack František Bartoš |
| author_facet | Ulrich Schimmack František Bartoš |
| author_sort | Ulrich Schimmack |
| collection | DOAJ |
| description | The influential claim that most published results are false raised concerns about the trustworthiness and integrity of science. Since then, there have been numerous attempts to examine the rate of false-positive results that have failed to settle this question empirically. Here we propose a new way to estimate the false positive risk and apply the method to the results of (randomized) clinical trials in top medical journals. Contrary to claims that most published results are false, we find that the traditional significance criterion of α = .05 produces a false positive risk of 13%. Adjusting α to.01 lowers the false positive risk to less than 5%. However, our method does provide clear evidence of publication bias that leads to inflated effect size estimates. These results provide a solid empirical foundation for evaluations of the trustworthiness of medical research. |
| first_indexed | 2024-03-12T02:32:29Z |
| format | Article |
| id | doaj.art-0aa40a211afb41e2bd334050defe4223 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-03-12T02:32:29Z |
| publishDate | 2023-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-0aa40a211afb41e2bd334050defe42232023-09-05T05:31:41ZengPublic Library of Science (PLoS)PLoS ONE1932-62032023-01-01188Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-valuesUlrich SchimmackFrantišek BartošThe influential claim that most published results are false raised concerns about the trustworthiness and integrity of science. Since then, there have been numerous attempts to examine the rate of false-positive results that have failed to settle this question empirically. Here we propose a new way to estimate the false positive risk and apply the method to the results of (randomized) clinical trials in top medical journals. Contrary to claims that most published results are false, we find that the traditional significance criterion of α = .05 produces a false positive risk of 13%. Adjusting α to.01 lowers the false positive risk to less than 5%. However, our method does provide clear evidence of publication bias that leads to inflated effect size estimates. These results provide a solid empirical foundation for evaluations of the trustworthiness of medical research.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468063/?tool=EBI |
| spellingShingle | Ulrich Schimmack František Bartoš Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values PLoS ONE |
| title | Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values |
| title_full | Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values |
| title_fullStr | Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values |
| title_full_unstemmed | Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values |
| title_short | Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values |
| title_sort | estimating the false discovery risk of randomized clinical trials in medical journals based on published p values |
| url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468063/?tool=EBI |
| work_keys_str_mv | AT ulrichschimmack estimatingthefalsediscoveryriskofrandomizedclinicaltrialsinmedicaljournalsbasedonpublishedpvalues AT frantisekbartos estimatingthefalsediscoveryriskofrandomizedclinicaltrialsinmedicaljournalsbasedonpublishedpvalues |