Sensitivity analysis for causality in observational studies for regulatory science
Abstract Objective: The United States Congress passed the 21st Century Cures Act mandating the development of Food and Drug Administration guidance on regulatory use of real-world evidence. The Forum on the Integration of Observational and Randomized Data conducted a meeting with various stakehold...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2023-01-01
|
| Series: | Journal of Clinical and Translational Science |
| Subjects: | |
| Online Access: | https://www.cambridge.org/core/product/identifier/S205986612300688X/type/journal_article |
| _version_ | 1797367825179869184 |
|---|---|
| author | Iván Díaz Hana Lee Emre Kıcıman Edward J. Schenck Mouna Akacha Dean Follman Debashis Ghosh |
| author_facet | Iván Díaz Hana Lee Emre Kıcıman Edward J. Schenck Mouna Akacha Dean Follman Debashis Ghosh |
| author_sort | Iván Díaz |
| collection | DOAJ |
| description |
Abstract
Objective:
The United States Congress passed the 21st Century Cures Act mandating the development of Food and Drug Administration guidance on regulatory use of real-world evidence. The Forum on the Integration of Observational and Randomized Data conducted a meeting with various stakeholder groups to build consensus around best practices for the use of real-world data (RWD) to support regulatory science. Our companion paper describes in detail the context and discussion of the meeting, which includes a recommendation to use a causal roadmap for study designs using RWD. This article discusses one step of the roadmap: the specification of a sensitivity analysis for testing robustness to violations of causal model assumptions.
Methods:
We present an example of a sensitivity analysis from a RWD study on the effectiveness of Nifurtimox in treating Chagas disease, and an overview of various methods, emphasizing practical considerations on their use for regulatory purposes.
Results:
Sensitivity analyses must be accompanied by careful design of other aspects of the causal roadmap. Their prespecification is crucial to avoid wrong conclusions due to researcher degrees of freedom. Sensitivity analysis methods require auxiliary information to produce meaningful conclusions; it is important that they have at least two properties: the validity of the conclusions does not rely on unverifiable assumptions, and the auxiliary information required by the method is learnable from the corpus of current scientific knowledge.
Conclusions:
Prespecified and assumption-lean sensitivity analyses are a crucial tool that can strengthen the validity and trustworthiness of effectiveness conclusions for regulatory science.
|
| first_indexed | 2024-03-08T17:23:38Z |
| format | Article |
| id | doaj.art-5386275642b143829954ba37150f6aa0 |
| institution | Directory Open Access Journal |
| issn | 2059-8661 |
| language | English |
| last_indexed | 2024-03-08T17:23:38Z |
| publishDate | 2023-01-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | Journal of Clinical and Translational Science |
| spelling | doaj.art-5386275642b143829954ba37150f6aa02024-01-03T02:24:48ZengCambridge University PressJournal of Clinical and Translational Science2059-86612023-01-01710.1017/cts.2023.688Sensitivity analysis for causality in observational studies for regulatory scienceIván Díaz0https://orcid.org/0000-0001-9056-2047Hana Lee1Emre Kıcıman2Edward J. Schenck3Mouna Akacha4Dean Follman5Debashis Ghosh6Division of Biostatistics, Department of Population Health, New York University Grossman School of Medicine, New York, NY, USAOffice of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USAMicrosoft Research, Redmond, WA, USADepartment of Medicine, Weill Cornell Medicine, New York, NY, USANovartis Pharma AG, Basel, SwitzerlandBiostatistics Research Branch, National Institute of Allergy and Infectious Disease, Silver Spring, MD, USADepartment of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Colorado, USA Abstract Objective: The United States Congress passed the 21st Century Cures Act mandating the development of Food and Drug Administration guidance on regulatory use of real-world evidence. The Forum on the Integration of Observational and Randomized Data conducted a meeting with various stakeholder groups to build consensus around best practices for the use of real-world data (RWD) to support regulatory science. Our companion paper describes in detail the context and discussion of the meeting, which includes a recommendation to use a causal roadmap for study designs using RWD. This article discusses one step of the roadmap: the specification of a sensitivity analysis for testing robustness to violations of causal model assumptions. Methods: We present an example of a sensitivity analysis from a RWD study on the effectiveness of Nifurtimox in treating Chagas disease, and an overview of various methods, emphasizing practical considerations on their use for regulatory purposes. Results: Sensitivity analyses must be accompanied by careful design of other aspects of the causal roadmap. Their prespecification is crucial to avoid wrong conclusions due to researcher degrees of freedom. Sensitivity analysis methods require auxiliary information to produce meaningful conclusions; it is important that they have at least two properties: the validity of the conclusions does not rely on unverifiable assumptions, and the auxiliary information required by the method is learnable from the corpus of current scientific knowledge. Conclusions: Prespecified and assumption-lean sensitivity analyses are a crucial tool that can strengthen the validity and trustworthiness of effectiveness conclusions for regulatory science. https://www.cambridge.org/core/product/identifier/S205986612300688X/type/journal_articleCausal inferencesensitivity analysisreal-world dataobservational datastudy design |
| spellingShingle | Iván Díaz Hana Lee Emre Kıcıman Edward J. Schenck Mouna Akacha Dean Follman Debashis Ghosh Sensitivity analysis for causality in observational studies for regulatory science Journal of Clinical and Translational Science Causal inference sensitivity analysis real-world data observational data study design |
| title | Sensitivity analysis for causality in observational studies for regulatory science |
| title_full | Sensitivity analysis for causality in observational studies for regulatory science |
| title_fullStr | Sensitivity analysis for causality in observational studies for regulatory science |
| title_full_unstemmed | Sensitivity analysis for causality in observational studies for regulatory science |
| title_short | Sensitivity analysis for causality in observational studies for regulatory science |
| title_sort | sensitivity analysis for causality in observational studies for regulatory science |
| topic | Causal inference sensitivity analysis real-world data observational data study design |
| url | https://www.cambridge.org/core/product/identifier/S205986612300688X/type/journal_article |
| work_keys_str_mv | AT ivandiaz sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience AT hanalee sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience AT emrekıcıman sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience AT edwardjschenck sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience AT mounaakacha sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience AT deanfollman sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience AT debashisghosh sensitivityanalysisforcausalityinobservationalstudiesforregulatoryscience |