An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions
Introduction: Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs and vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternat...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-03-01
|
| Series: | Frontiers in Pharmacology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fphar.2023.1064567/full |
| _version_ | 1827986498854584320 |
|---|---|
| author | Minhan Dai Luis Furuya-Kanamori Asma Syed Lifeng Lin Qiang Wang |
| author_facet | Minhan Dai Luis Furuya-Kanamori Asma Syed Lifeng Lin Qiang Wang |
| author_sort | Minhan Dai |
| collection | DOAJ |
| description | Introduction: Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs and vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternative source for safety assessment. In this study, we aimed to investigate whether there is any difference between RCTs and NRSIs in the evaluation of adverse events.Methods: We used the dataset of systematic reviews with at least one meta-analysis including both RCTs and NRSIs and collected the 2 × 2 table information (i.e., numbers of cases and sample sizes in intervention and control groups) of each study in the meta-analysis. We matched RCTs and NRSIs by their sample sizes (ratio: 0.85/1 to 1/0.85) within a meta-analysis. We estimated the ratio of the odds ratios (RORs) of an NRSI against an RCT in each pair and used the inverse variance as the weight to combine the natural logarithm of ROR (lnROR).Results: We included systematic reviews with 178 meta analyses, from which we confirmed 119 pairs of RCTs and NRSIs. The pooled ROR of NRSIs compared to that of RCTs was estimated to be 0.96 (95% confidence interval: 0.87 and 1.07). Similar results were obtained with different sample size subgroups and treatment subgroups. With the increase in sample size, the difference in ROR between RCTs and NRSIs decreased, although not significantly.Discussion: There was no substantial difference in the effects between RCTs and NRSIs in safety assessment when they have similar sample sizes. Evidence from NRSIs might be considered a supplement to RCTs for safety assessment. |
| first_indexed | 2024-04-09T23:32:19Z |
| format | Article |
| id | doaj.art-db3c088cb6504b0894be40ae3876c71f |
| institution | Directory Open Access Journal |
| issn | 1663-9812 |
| language | English |
| last_indexed | 2024-04-09T23:32:19Z |
| publishDate | 2023-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Pharmacology |
| spelling | doaj.art-db3c088cb6504b0894be40ae3876c71f2023-03-21T04:59:22ZengFrontiers Media S.A.Frontiers in Pharmacology1663-98122023-03-011410.3389/fphar.2023.10645671064567An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventionsMinhan Dai0Luis Furuya-Kanamori1Asma Syed2Lifeng Lin3Qiang Wang4Mental Health Center, West China Hospital of Sichuan University, Chengdu, ChinaSchool of Public Health, Faculty of Medicine, The University of Queensland, Herston, QL, AustraliaDepartment of Population Medicine, College of Medicine, Qatar University, Doha, QatarDepartment of Epidemiology and Biostatistics, University of Arizona, Tucson, AZ, United StatesMental Health Center, West China Hospital of Sichuan University, Chengdu, ChinaIntroduction: Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs and vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternative source for safety assessment. In this study, we aimed to investigate whether there is any difference between RCTs and NRSIs in the evaluation of adverse events.Methods: We used the dataset of systematic reviews with at least one meta-analysis including both RCTs and NRSIs and collected the 2 × 2 table information (i.e., numbers of cases and sample sizes in intervention and control groups) of each study in the meta-analysis. We matched RCTs and NRSIs by their sample sizes (ratio: 0.85/1 to 1/0.85) within a meta-analysis. We estimated the ratio of the odds ratios (RORs) of an NRSI against an RCT in each pair and used the inverse variance as the weight to combine the natural logarithm of ROR (lnROR).Results: We included systematic reviews with 178 meta analyses, from which we confirmed 119 pairs of RCTs and NRSIs. The pooled ROR of NRSIs compared to that of RCTs was estimated to be 0.96 (95% confidence interval: 0.87 and 1.07). Similar results were obtained with different sample size subgroups and treatment subgroups. With the increase in sample size, the difference in ROR between RCTs and NRSIs decreased, although not significantly.Discussion: There was no substantial difference in the effects between RCTs and NRSIs in safety assessment when they have similar sample sizes. Evidence from NRSIs might be considered a supplement to RCTs for safety assessment.https://www.frontiersin.org/articles/10.3389/fphar.2023.1064567/fullrandomized controlled trialnon-randomized studies of interventionadverse eventsharmful effectempirical comparison |
| spellingShingle | Minhan Dai Luis Furuya-Kanamori Asma Syed Lifeng Lin Qiang Wang An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions Frontiers in Pharmacology randomized controlled trial non-randomized studies of intervention adverse events harmful effect empirical comparison |
| title | An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions |
| title_full | An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions |
| title_fullStr | An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions |
| title_full_unstemmed | An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions |
| title_short | An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions |
| title_sort | empirical comparison of the harmful effects for randomized controlled trials and non randomized studies of interventions |
| topic | randomized controlled trial non-randomized studies of intervention adverse events harmful effect empirical comparison |
| url | https://www.frontiersin.org/articles/10.3389/fphar.2023.1064567/full |
| work_keys_str_mv | AT minhandai anempiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT luisfuruyakanamori anempiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT asmasyed anempiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT lifenglin anempiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT qiangwang anempiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT minhandai empiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT luisfuruyakanamori empiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT asmasyed empiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT lifenglin empiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions AT qiangwang empiricalcomparisonoftheharmfuleffectsforrandomizedcontrolledtrialsandnonrandomizedstudiesofinterventions |