Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study
Abstract Background Network meta-analyses can be valuable for decision-makers in guiding clinical practice. However, for network meta-analysis results to be reliable, the assumptions of both transitivity and coherence must be met, and the methodology should adhere to current best practices. We aimed...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-01-01
|
| Series: | BMC Musculoskeletal Disorders |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12891-023-07119-w |
| _version_ | 1797363723105468416 |
|---|---|
| author | Nicolai Sandau Thomas Vedste Aagaard Asbjørn Hróbjartsson Ian A. Harris Stig Brorson |
| author_facet | Nicolai Sandau Thomas Vedste Aagaard Asbjørn Hróbjartsson Ian A. Harris Stig Brorson |
| author_sort | Nicolai Sandau |
| collection | DOAJ |
| description | Abstract Background Network meta-analyses can be valuable for decision-makers in guiding clinical practice. However, for network meta-analysis results to be reliable, the assumptions of both transitivity and coherence must be met, and the methodology should adhere to current best practices. We aimed to assess whether network meta-analyses of randomized controlled trials (RCTs) comparing interventions for proximal humerus fractures provide reliable estimates of intervention effects. Methods We searched PubMed, EMBASE, The Cochrane Library, and Web of Science for network meta-analyses comparing interventions for proximal humerus fractures. We critically assessed the methodology regarding the development of a protocol, search strategy, trial inclusion, outcome extraction, and the methods used to conduct the network meta-analyses. We assessed the transitivity and coherence of the network graphs for the Constant score (CS), Disabilities of the Arm, Shoulder, and Hand score (DASH), and additional surgery. Transitivity was assessed by comparing probable effect modifiers (age, gender, fracture morphology, and comorbidities) across intervention comparisons. Coherence was assessed using Separating Indirect from Direct Evidence (SIDE) (Separating Indirect from Direct Evidence) and the design-by-treatment interaction test. We used CINeMA (Confidence in Network Meta-analyses) to assess the confidence in the results. Results None of the three included network meta-analyses had a publicly available protocol or data-analysis plan, and they all had methodological flaws that could threaten the validity of their results. Although we did not detect incoherence for most comparisons, the transitivity assumption was violated for CS, DASH, and additional surgery in all three network meta-analyses. Additionally, the confidence in the results was ‘very low’ primarily due to within-study bias, reporting bias, intransitivity, imprecision, and heterogeneity. Conclusions Current network meta-analyses of RCTs comparing interventions for proximal humerus fractures do not provide reliable estimates of intervention effects. We advise caution in using these network meta-analyses to guide clinical practice. To improve the utility of network meta-analyses to guide clinical practice, journal editors should require that network meta-analyses are done according to a predefined analysis plan in a publicly available protocol and that both coherence and transitivity have been adequately assessed and reported. |
| first_indexed | 2024-03-08T16:25:15Z |
| format | Article |
| id | doaj.art-3c6992fc66514da2b64c18d36813d6e6 |
| institution | Directory Open Access Journal |
| issn | 1471-2474 |
| language | English |
| last_indexed | 2024-03-08T16:25:15Z |
| publishDate | 2024-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Musculoskeletal Disorders |
| spelling | doaj.art-3c6992fc66514da2b64c18d36813d6e62024-01-07T12:04:11ZengBMCBMC Musculoskeletal Disorders1471-24742024-01-0125111210.1186/s12891-023-07119-wTransitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological studyNicolai Sandau0Thomas Vedste Aagaard1Asbjørn Hróbjartsson2Ian A. Harris3Stig Brorson4Centre for Evidence-Based Orthopedics, Department of Orthopedic Surgery, Zealand University HospitalThe Research and Implementation Unit PROgrez, Department of Physiotherapy and Occupational Therapy, Naestved-Slagelse-Ringsted HospitalsCentre for Evidence-Based Medicine Odense (CEBMO), and Cochrane Denmark, Department of Clinical Research, University of Southern DenmarkWhitlam Orthopaedic Research Centre, Ingham Institute for Applied Medical Research, South Western Sydney Clinical School, University of New South Wales (UNSW Sydney)Centre for Evidence-Based Orthopedics, Department of Orthopedic Surgery, Zealand University HospitalAbstract Background Network meta-analyses can be valuable for decision-makers in guiding clinical practice. However, for network meta-analysis results to be reliable, the assumptions of both transitivity and coherence must be met, and the methodology should adhere to current best practices. We aimed to assess whether network meta-analyses of randomized controlled trials (RCTs) comparing interventions for proximal humerus fractures provide reliable estimates of intervention effects. Methods We searched PubMed, EMBASE, The Cochrane Library, and Web of Science for network meta-analyses comparing interventions for proximal humerus fractures. We critically assessed the methodology regarding the development of a protocol, search strategy, trial inclusion, outcome extraction, and the methods used to conduct the network meta-analyses. We assessed the transitivity and coherence of the network graphs for the Constant score (CS), Disabilities of the Arm, Shoulder, and Hand score (DASH), and additional surgery. Transitivity was assessed by comparing probable effect modifiers (age, gender, fracture morphology, and comorbidities) across intervention comparisons. Coherence was assessed using Separating Indirect from Direct Evidence (SIDE) (Separating Indirect from Direct Evidence) and the design-by-treatment interaction test. We used CINeMA (Confidence in Network Meta-analyses) to assess the confidence in the results. Results None of the three included network meta-analyses had a publicly available protocol or data-analysis plan, and they all had methodological flaws that could threaten the validity of their results. Although we did not detect incoherence for most comparisons, the transitivity assumption was violated for CS, DASH, and additional surgery in all three network meta-analyses. Additionally, the confidence in the results was ‘very low’ primarily due to within-study bias, reporting bias, intransitivity, imprecision, and heterogeneity. Conclusions Current network meta-analyses of RCTs comparing interventions for proximal humerus fractures do not provide reliable estimates of intervention effects. We advise caution in using these network meta-analyses to guide clinical practice. To improve the utility of network meta-analyses to guide clinical practice, journal editors should require that network meta-analyses are done according to a predefined analysis plan in a publicly available protocol and that both coherence and transitivity have been adequately assessed and reported.https://doi.org/10.1186/s12891-023-07119-wProximal humerus fracturesShoulder fracturesNetwork meta-analysesMethodological qualityTransitivityIndirectness |
| spellingShingle | Nicolai Sandau Thomas Vedste Aagaard Asbjørn Hróbjartsson Ian A. Harris Stig Brorson Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study BMC Musculoskeletal Disorders Proximal humerus fractures Shoulder fractures Network meta-analyses Methodological quality Transitivity Indirectness |
| title | Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study |
| title_full | Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study |
| title_fullStr | Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study |
| title_full_unstemmed | Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study |
| title_short | Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study |
| title_sort | transitivity coherence and reliability of network meta analyses comparing proximal humerus fracture treatments a meta epidemiological study |
| topic | Proximal humerus fractures Shoulder fractures Network meta-analyses Methodological quality Transitivity Indirectness |
| url | https://doi.org/10.1186/s12891-023-07119-w |
| work_keys_str_mv | AT nicolaisandau transitivitycoherenceandreliabilityofnetworkmetaanalysescomparingproximalhumerusfracturetreatmentsametaepidemiologicalstudy AT thomasvedsteaagaard transitivitycoherenceandreliabilityofnetworkmetaanalysescomparingproximalhumerusfracturetreatmentsametaepidemiologicalstudy AT asbjørnhrobjartsson transitivitycoherenceandreliabilityofnetworkmetaanalysescomparingproximalhumerusfracturetreatmentsametaepidemiologicalstudy AT ianaharris transitivitycoherenceandreliabilityofnetworkmetaanalysescomparingproximalhumerusfracturetreatmentsametaepidemiologicalstudy AT stigbrorson transitivitycoherenceandreliabilityofnetworkmetaanalysescomparingproximalhumerusfracturetreatmentsametaepidemiologicalstudy |