Post-hoc power analysis: a conceptually valid approach for power based on observed study data
Power analysis is a key component of planning prospective studies such as clinical trials. However, some journals in biomedical and psychosocial sciences request power analysis for data already collected and analysed before accepting manuscripts for publication. Many have raised concerns about the c...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2022-09-01
|
| Series: | General Psychiatry |
| Online Access: | https://gpsych.bmj.com/content/35/4/e100764.full |
| _version_ | 1798030406323273728 |
|---|---|
| author | Xin M Tu Kun Yang Justin Tu Ruohui Chen Xinlian Zhang Natalie E Quach |
| author_facet | Xin M Tu Kun Yang Justin Tu Ruohui Chen Xinlian Zhang Natalie E Quach |
| author_sort | Xin M Tu |
| collection | DOAJ |
| description | Power analysis is a key component of planning prospective studies such as clinical trials. However, some journals in biomedical and psychosocial sciences request power analysis for data already collected and analysed before accepting manuscripts for publication. Many have raised concerns about the conceptual basis for such post-hoc power analyses. More recently, Zhang et al showed by using simulation studies that such power analyses do not indicate true power for detecting statistical significance since post-hoc power estimates vary in the range of practical interests and can be very different from the true power. On the other hand, journals’ request for information about the reliability of statistical findings in a manuscript due to small sample sizes is justified since the sample size plays an important role in the reproducibility of statistical findings. The problem is the wording of the journals' request, as the current power analysis paradigm is not designed to address journals’ concerns about the reliability of the statistical findings. In this paper, we propose an alternate formulation of power analysis to provide a conceptually valid approach to the journals’ wrongly worded but practically significant concern. |
| first_indexed | 2024-04-11T19:40:53Z |
| format | Article |
| id | doaj.art-a6b1ca0e05b142a4b30b9a831151a9ae |
| institution | Directory Open Access Journal |
| issn | 2517-729X |
| language | English |
| last_indexed | 2024-04-11T19:40:53Z |
| publishDate | 2022-09-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | General Psychiatry |
| spelling | doaj.art-a6b1ca0e05b142a4b30b9a831151a9ae2022-12-22T04:06:43ZengBMJ Publishing GroupGeneral Psychiatry2517-729X2022-09-0135410.1136/gpsych-2022-100764Post-hoc power analysis: a conceptually valid approach for power based on observed study dataXin M Tu0Kun Yang1Justin Tu2Ruohui Chen3Xinlian Zhang4Natalie E Quach51 Division of Biostatistics and Bioinformatics, Herbert Wertheim School of Public Health and Human Longevity Science, UC San Diego, La Jolla, California, USAEmergency Department, The First People’s Hospital of Longquanyi District, Chengdu, Sichuan, China2 Department of Orthopedics, Emory Healthcare, Emory University, Atlanta, Georgia, USA1 Division of Biostatistics and Bioinformatics, Herbert Wertheim School of Public Health and Human Longevity Science, UC San Diego, La Jolla, California, USA1 Division of Biostatistics and Bioinformatics, Herbert Wertheim School of Public Health and Human Longevity Science, UC San Diego, La Jolla, California, USA1 Division of Biostatistics and Bioinformatics, Herbert Wertheim School of Public Health and Human Longevity Science, UC San Diego, La Jolla, California, USAPower analysis is a key component of planning prospective studies such as clinical trials. However, some journals in biomedical and psychosocial sciences request power analysis for data already collected and analysed before accepting manuscripts for publication. Many have raised concerns about the conceptual basis for such post-hoc power analyses. More recently, Zhang et al showed by using simulation studies that such power analyses do not indicate true power for detecting statistical significance since post-hoc power estimates vary in the range of practical interests and can be very different from the true power. On the other hand, journals’ request for information about the reliability of statistical findings in a manuscript due to small sample sizes is justified since the sample size plays an important role in the reproducibility of statistical findings. The problem is the wording of the journals' request, as the current power analysis paradigm is not designed to address journals’ concerns about the reliability of the statistical findings. In this paper, we propose an alternate formulation of power analysis to provide a conceptually valid approach to the journals’ wrongly worded but practically significant concern.https://gpsych.bmj.com/content/35/4/e100764.full |
| spellingShingle | Xin M Tu Kun Yang Justin Tu Ruohui Chen Xinlian Zhang Natalie E Quach Post-hoc power analysis: a conceptually valid approach for power based on observed study data General Psychiatry |
| title | Post-hoc power analysis: a conceptually valid approach for power based on observed study data |
| title_full | Post-hoc power analysis: a conceptually valid approach for power based on observed study data |
| title_fullStr | Post-hoc power analysis: a conceptually valid approach for power based on observed study data |
| title_full_unstemmed | Post-hoc power analysis: a conceptually valid approach for power based on observed study data |
| title_short | Post-hoc power analysis: a conceptually valid approach for power based on observed study data |
| title_sort | post hoc power analysis a conceptually valid approach for power based on observed study data |
| url | https://gpsych.bmj.com/content/35/4/e100764.full |
| work_keys_str_mv | AT xinmtu posthocpoweranalysisaconceptuallyvalidapproachforpowerbasedonobservedstudydata AT kunyang posthocpoweranalysisaconceptuallyvalidapproachforpowerbasedonobservedstudydata AT justintu posthocpoweranalysisaconceptuallyvalidapproachforpowerbasedonobservedstudydata AT ruohuichen posthocpoweranalysisaconceptuallyvalidapproachforpowerbasedonobservedstudydata AT xinlianzhang posthocpoweranalysisaconceptuallyvalidapproachforpowerbasedonobservedstudydata AT natalieequach posthocpoweranalysisaconceptuallyvalidapproachforpowerbasedonobservedstudydata |