Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery
Background A recent study concluded that most findings reported as significant in sports medicine and arthroscopic surgery are not “robust” when evaluated with the Fragility Index (FI). A secondary analysis of data from a previous study was performed to investigate (1) the correctness of the finding...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2019-05-01
|
| Series: | PeerJ |
| Subjects: | |
| Online Access: | https://peerj.com/articles/6813.pdf |
| _version_ | 1797424076977864704 |
|---|---|
| author | Aleksi Reito Lauri Raittio Olli Helminen |
| author_facet | Aleksi Reito Lauri Raittio Olli Helminen |
| author_sort | Aleksi Reito |
| collection | DOAJ |
| description | Background A recent study concluded that most findings reported as significant in sports medicine and arthroscopic surgery are not “robust” when evaluated with the Fragility Index (FI). A secondary analysis of data from a previous study was performed to investigate (1) the correctness of the findings, (2) the association between FI, p-value and post hoc power, (3) median power to detect a medium effect size, and (4) the implementation of sample size analysis in these randomized controlled trials (RCTs). Methods In addition to the 48 studies listed in the appendix accompanying the original study by Khan et al. (2017) we did a follow-up literature search and 18 additional studies were found. In total 66 studies were included in the analysis. We calculated post hoc power, p-values and confidence intervals associated with the main outcome variable. Use of a priori power analysis was recorded. The median power to detect small (h > 0.2), medium (h > 0.5), or large effect (h > 0.8) with a baseline proportion of events of 10% and 30% in each study included was calculated. Three simulation data sets were used to validate our findings. Results Inconsistencies were found in eight studies. A priori power analysis was missing in one-fourth of studies (16/66). The median power to detect a medium effect size with a baseline proportion of events of 10% and 30% was 42% and 43%, respectively. The FI was inherently associated with the achieved p-value and post hoc power. Discussion A relatively high proportion of studies had inconsistencies. The FI is a surrogate measure for p-value and post hoc power. Based on these studies, the median power in this field of research is suboptimal. There is an urgent need to investigate how well research claims in orthopedics hold in a replicated setting and the validity of research findings. |
| first_indexed | 2024-03-09T07:56:05Z |
| format | Article |
| id | doaj.art-dd0fd45ae3314e28af50a7ac982eb195 |
| institution | Directory Open Access Journal |
| issn | 2167-8359 |
| language | English |
| last_indexed | 2024-03-09T07:56:05Z |
| publishDate | 2019-05-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ |
| spelling | doaj.art-dd0fd45ae3314e28af50a7ac982eb1952023-12-03T01:00:50ZengPeerJ Inc.PeerJ2167-83592019-05-017e681310.7717/peerj.6813Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgeryAleksi Reito0Lauri Raittio1Olli Helminen2Department of Surgery, Central Finland Hospital, Jyväskylä, Keski-Suomi, FinlandMedical School, University of Tampere, Tampere, FinlandDepartment of Surgery, Central Finland Hospital, Jyväskylä, Keski-Suomi, FinlandBackground A recent study concluded that most findings reported as significant in sports medicine and arthroscopic surgery are not “robust” when evaluated with the Fragility Index (FI). A secondary analysis of data from a previous study was performed to investigate (1) the correctness of the findings, (2) the association between FI, p-value and post hoc power, (3) median power to detect a medium effect size, and (4) the implementation of sample size analysis in these randomized controlled trials (RCTs). Methods In addition to the 48 studies listed in the appendix accompanying the original study by Khan et al. (2017) we did a follow-up literature search and 18 additional studies were found. In total 66 studies were included in the analysis. We calculated post hoc power, p-values and confidence intervals associated with the main outcome variable. Use of a priori power analysis was recorded. The median power to detect small (h > 0.2), medium (h > 0.5), or large effect (h > 0.8) with a baseline proportion of events of 10% and 30% in each study included was calculated. Three simulation data sets were used to validate our findings. Results Inconsistencies were found in eight studies. A priori power analysis was missing in one-fourth of studies (16/66). The median power to detect a medium effect size with a baseline proportion of events of 10% and 30% was 42% and 43%, respectively. The FI was inherently associated with the achieved p-value and post hoc power. Discussion A relatively high proportion of studies had inconsistencies. The FI is a surrogate measure for p-value and post hoc power. Based on these studies, the median power in this field of research is suboptimal. There is an urgent need to investigate how well research claims in orthopedics hold in a replicated setting and the validity of research findings.https://peerj.com/articles/6813.pdfPower analysisEffect sizeStatistical powerReplicationReproducibility |
| spellingShingle | Aleksi Reito Lauri Raittio Olli Helminen Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery PeerJ Power analysis Effect size Statistical power Replication Reproducibility |
| title | Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery |
| title_full | Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery |
| title_fullStr | Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery |
| title_full_unstemmed | Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery |
| title_short | Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery |
| title_sort | fragility index power strength and robustness of findings in sports medicine and arthroscopic surgery a secondary analysis of data from a study on use of the fragility index in sports surgery |
| topic | Power analysis Effect size Statistical power Replication Reproducibility |
| url | https://peerj.com/articles/6813.pdf |
| work_keys_str_mv | AT aleksireito fragilityindexpowerstrengthandrobustnessoffindingsinsportsmedicineandarthroscopicsurgeryasecondaryanalysisofdatafromastudyonuseofthefragilityindexinsportssurgery AT lauriraittio fragilityindexpowerstrengthandrobustnessoffindingsinsportsmedicineandarthroscopicsurgeryasecondaryanalysisofdatafromastudyonuseofthefragilityindexinsportssurgery AT ollihelminen fragilityindexpowerstrengthandrobustnessoffindingsinsportsmedicineandarthroscopicsurgeryasecondaryanalysisofdatafromastudyonuseofthefragilityindexinsportssurgery |