Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study. - Nikolakopoulou, A; Mavridis, D; Furukawa, TA; Cipriani, A; Tricco, AC; Straus, SE; Siontis, GCM; Egger, M; Salanti, G (BMJ Publishing Group, 2018)
Objectives To examine whether the continuous updating of networks of prospectively planned randomised controlled trials (RCTs) (‘living’ network meta-analysis) provides strong evidence against the null hypothesis in comparative effectiveness of medical interventions earlier than the updating of conventional, pairwise meta-analysis. Design Empirical study of the accumulating evidence about the comparative effectiveness of clinical interventions. Data Sources Database of network meta-analyses of RCTs identified through searches of MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews until April 14, 2015. Eligibility criteria for study selection Network meta-analyses published after January 2012 that compared at least five treatments and included at least 20 RCTs. Clinical experts were asked to identify in each network the treatment comparisons of greatest clinical interest. We excluded comparisons for which direct and indirect evidence disagreed, based on side-splitting test P<0.10. Outcomes and analysis For each selected comparison we performed cumulative pairwise and network meta-analyses. We constructed monitoring boundaries of statistical significance and considered the evidence against the null hypothesis as strong when the monitoring boundaries were crossed. We defined a significance level α=5%, power of 90% (β=10%) and an anticipated treatment effect to detect equal to the final estimate from the network meta-analysis. We compared the frequency and time to strong evidence against the null hypothesis between pairwise and network meta-analyses. Results Forty-nine comparisons of interest from 44 networks were included; the majority (n=39, 80%) were between active drugs, mainly from cardiology, endocrinology, psychiatry and rheumatology. Twenty-nine comparisons were informed by both direct and indirect evidence (59%), 13 comparisons by indirect (27%) and seven by direct evidence (14%) only. Both network and pairwise meta-analysis provided strong evidence against the null hypothesis for seven comparisons but for an additional ten comparisons only network meta-analysis provided strong evidence against the null hypothesis (P=0.002). The median time to strong evidence against the null hypothesis was 19 years with living network meta-analysis and 23 years with living pairwise meta-analysis (hazard ratio 2.78 (95% CI 1.00 to 7.72), P=0.05). Studies directly comparing the treatments of interest continued to be published for 8 comparisons after strong evidence had become evident in network meta-analysis. Conclusions In comparative effectiveness research, prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often and earlier than conventional, pairwise meta-analyses.