The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials
Abstract Background Phase III randomized controlled trials (RCT) typically exclude certain patient subgroups, thereby potentially jeopardizing estimation of a drug’s effects when prescribed to wider populations and under routine care (“effectiveness”). Conversely, enrolling heterogeneous populations...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2018-07-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12874-018-0534-6 |
| _version_ | 1818527690194419712 |
|---|---|
| author | Helene Karcher Shuai Fu Jie Meng Mikkel Zöllner Ankarfeldt Orestis Efthimiou Mark Belger Josep Maria Haro Lucien Abenhaim Clementine Nordon on behalf of the GetReal Consortium Work Package 2 |
| author_facet | Helene Karcher Shuai Fu Jie Meng Mikkel Zöllner Ankarfeldt Orestis Efthimiou Mark Belger Josep Maria Haro Lucien Abenhaim Clementine Nordon on behalf of the GetReal Consortium Work Package 2 |
| author_sort | Helene Karcher |
| collection | DOAJ |
| description | Abstract Background Phase III randomized controlled trials (RCT) typically exclude certain patient subgroups, thereby potentially jeopardizing estimation of a drug’s effects when prescribed to wider populations and under routine care (“effectiveness”). Conversely, enrolling heterogeneous populations in RCTs can increase endpoint variability and compromise detection of a drug’s effect. We developed the “RCT augmentation” method to quantitatively support RCT design in the identification of exclusion criteria to relax to address both of these considerations. In the present manuscript, we describe the method and a case study in schizophrenia. Methods We applied typical RCT exclusion criteria in a real-world dataset (cohort) of schizophrenia patients to define the “RCT population” subgroup, and assessed the impact of re-including each of the following patient subgroups: (1) illness duration 1–3 years; (2) suicide attempt; (3) alcohol abuse; (4) substance abuse; and (5) private practice management. Predictive models were built using data from different “augmented RCT populations” (i.e., subgroups where patients with one or two of such characteristics were re-included) to estimate the absolute effectiveness of the two most prevalent antipsychotics against real-world results from the entire cohort. Concurrently, the impact on RCT results of relaxing exclusion criteria was evaluated by calculating the comparative efficacy of those two antipsychotics in virtual RCTs drawing on different “augmented RCT populations”. Results Data from the “RCT population”, which was defined with typical exclusion criteria, allowed for a prediction of effectiveness with a bias < 2% and mean squared error (MSE) = 5.8–6.8%. Compared to this typical RCT, RCTs using augmented populations provided improved effectiveness predictions (bias < 2%, MSE = 5.3–6.7%), while returning more variable comparative effects. The impact of augmentation depended on the exclusion criterion relaxed. Furthermore, half of the benefit of relaxing each criterion was gained from re-including the first 10–20% of patients with the corresponding real-world characteristic. Conclusions Simulating the inclusion of real-world subpopulations into an RCT before running it allows for quantification of the impact of each re-inclusion upon effect detection (statistical power) and generalizability of trial results, thereby explicating this trade-off and enabling a controlled increase in population heterogeneity in the RCT design. |
| first_indexed | 2024-12-11T06:39:29Z |
| format | Article |
| id | doaj.art-476787d5e1b34f6b872ca637c030a6cc |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-12-11T06:39:29Z |
| publishDate | 2018-07-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-476787d5e1b34f6b872ca637c030a6cc2022-12-22T01:17:17ZengBMCBMC Medical Research Methodology1471-22882018-07-0118111410.1186/s12874-018-0534-6The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trialsHelene Karcher0Shuai Fu1Jie Meng2Mikkel Zöllner Ankarfeldt3Orestis Efthimiou4Mark Belger5Josep Maria Haro6Lucien Abenhaim7Clementine Nordon8on behalf of the GetReal Consortium Work Package 2Analytica LaserAnalytica LaserAnalytica LaserNovo Nordisk A/SDepartment of Hygiene and Epidemiology, University of Ioannina School of MedicineEli Lilly and Company, Lilly Research CentreParc Sanitari Sant Joan de Déu, CIBERSAM, Universitat de BarcelonaAnalytica LaserLASER CoreAbstract Background Phase III randomized controlled trials (RCT) typically exclude certain patient subgroups, thereby potentially jeopardizing estimation of a drug’s effects when prescribed to wider populations and under routine care (“effectiveness”). Conversely, enrolling heterogeneous populations in RCTs can increase endpoint variability and compromise detection of a drug’s effect. We developed the “RCT augmentation” method to quantitatively support RCT design in the identification of exclusion criteria to relax to address both of these considerations. In the present manuscript, we describe the method and a case study in schizophrenia. Methods We applied typical RCT exclusion criteria in a real-world dataset (cohort) of schizophrenia patients to define the “RCT population” subgroup, and assessed the impact of re-including each of the following patient subgroups: (1) illness duration 1–3 years; (2) suicide attempt; (3) alcohol abuse; (4) substance abuse; and (5) private practice management. Predictive models were built using data from different “augmented RCT populations” (i.e., subgroups where patients with one or two of such characteristics were re-included) to estimate the absolute effectiveness of the two most prevalent antipsychotics against real-world results from the entire cohort. Concurrently, the impact on RCT results of relaxing exclusion criteria was evaluated by calculating the comparative efficacy of those two antipsychotics in virtual RCTs drawing on different “augmented RCT populations”. Results Data from the “RCT population”, which was defined with typical exclusion criteria, allowed for a prediction of effectiveness with a bias < 2% and mean squared error (MSE) = 5.8–6.8%. Compared to this typical RCT, RCTs using augmented populations provided improved effectiveness predictions (bias < 2%, MSE = 5.3–6.7%), while returning more variable comparative effects. The impact of augmentation depended on the exclusion criterion relaxed. Furthermore, half of the benefit of relaxing each criterion was gained from re-including the first 10–20% of patients with the corresponding real-world characteristic. Conclusions Simulating the inclusion of real-world subpopulations into an RCT before running it allows for quantification of the impact of each re-inclusion upon effect detection (statistical power) and generalizability of trial results, thereby explicating this trade-off and enabling a controlled increase in population heterogeneity in the RCT design.http://link.springer.com/article/10.1186/s12874-018-0534-6Patient heterogeneityExternal validityReal-worldPragmatic trialsClinical drug developmentOptimal trial design |
| spellingShingle | Helene Karcher Shuai Fu Jie Meng Mikkel Zöllner Ankarfeldt Orestis Efthimiou Mark Belger Josep Maria Haro Lucien Abenhaim Clementine Nordon on behalf of the GetReal Consortium Work Package 2 The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials BMC Medical Research Methodology Patient heterogeneity External validity Real-world Pragmatic trials Clinical drug development Optimal trial design |
| title | The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials |
| title_full | The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials |
| title_fullStr | The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials |
| title_full_unstemmed | The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials |
| title_short | The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials |
| title_sort | rct augmentation a novel simulation method to add patient heterogeneity into phase iii trials |
| topic | Patient heterogeneity External validity Real-world Pragmatic trials Clinical drug development Optimal trial design |
| url | http://link.springer.com/article/10.1186/s12874-018-0534-6 |
| work_keys_str_mv | AT helenekarcher therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT shuaifu therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT jiemeng therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT mikkelzollnerankarfeldt therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT orestisefthimiou therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT markbelger therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT josepmariaharo therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT lucienabenhaim therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT clementinenordon therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT onbehalfofthegetrealconsortiumworkpackage2 therctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT helenekarcher rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT shuaifu rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT jiemeng rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT mikkelzollnerankarfeldt rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT orestisefthimiou rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT markbelger rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT josepmariaharo rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT lucienabenhaim rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT clementinenordon rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials AT onbehalfofthegetrealconsortiumworkpackage2 rctaugmentationanovelsimulationmethodtoaddpatientheterogeneityintophaseiiitrials |