Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies
Introduction and objective: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Envir...
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2019-09-01
|
| Series: | Environment International |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S0160412018327302 |
| _version_ | 1818133422722252800 |
|---|---|
| author | Elizabeth G. Radke Barbara Glenn Audrey Galizia Amanda Persad Rebecca Nachman Thomas Bateson J. Michael Wright Ana Navas-Acien Whitney D. Arroyave Robin C. Puett Emily W. Harville Anna Z. Pollack Jane S. Burns Courtney D. Lynch Sharon K. Sagiv Cheryl Stein Glinda S. Cooper |
| author_facet | Elizabeth G. Radke Barbara Glenn Audrey Galizia Amanda Persad Rebecca Nachman Thomas Bateson J. Michael Wright Ana Navas-Acien Whitney D. Arroyave Robin C. Puett Emily W. Harville Anna Z. Pollack Jane S. Burns Courtney D. Lynch Sharon K. Sagiv Cheryl Stein Glinda S. Cooper |
| author_sort | Elizabeth G. Radke |
| collection | DOAJ |
| description | Introduction and objective: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria. Methods: We used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposure-specific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard). Results: We discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders. Conclusions: Outcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed. |
| first_indexed | 2024-12-11T08:52:29Z |
| format | Article |
| id | doaj.art-339768571b8041e18952c3c9936e8074 |
| institution | Directory Open Access Journal |
| issn | 0160-4120 |
| language | English |
| last_indexed | 2024-12-11T08:52:29Z |
| publishDate | 2019-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Environment International |
| spelling | doaj.art-339768571b8041e18952c3c9936e80742022-12-22T01:13:59ZengElsevierEnvironment International0160-41202019-09-01130Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studiesElizabeth G. Radke0Barbara Glenn1Audrey Galizia2Amanda Persad3Rebecca Nachman4Thomas Bateson5J. Michael Wright6Ana Navas-Acien7Whitney D. Arroyave8Robin C. Puett9Emily W. Harville10Anna Z. Pollack11Jane S. Burns12Courtney D. Lynch13Sharon K. Sagiv14Cheryl Stein15Glinda S. Cooper16U.S. Environmental Protection Agency, National Center for Environmental Assessment, United States; Corresponding author.U.S. Environmental Protection Agency, National Center for Environmental Assessment, United StatesU.S. Environmental Protection Agency, National Center for Environmental Assessment, United StatesU.S. Environmental Protection Agency, National Center for Environmental Assessment, United StatesU.S. Environmental Protection Agency, National Center for Environmental Assessment, United StatesU.S. Environmental Protection Agency, National Center for Environmental Assessment, United StatesU.S. Environmental Protection Agency, National Center for Environmental Assessment, United StatesDepartment of Environmental Health Sciences, Columbia University Mailman School of Public Health, United StatesIntegrated Laboratory Systems, United StatesDepartment of Epidemiology and Biostatistics, University of Maryland School of Public Health, United StatesDepartment of Epidemiology, Tulane University School of Public Health and Tropical Medicine, United StatesDepartment of Global and Community Health, College of Health and Human Services, George Mason University, United StatesDepartment of Environmental Health, Harvard T. H. Chan School of Public Health, United StatesDepartment of Obstetrics and Gynecology, The Ohio State University College of Medicine, United StatesDivision of Epidemiology, University of California Berkeley, United StatesDepartment of Child and Adolescent Psychiatry, Hassenfeld Children's Hospital at NYU Langone, United StatesU.S. Environmental Protection Agency, National Center for Environmental Assessment, United States; The Innocence Project, United StatesIntroduction and objective: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria. Methods: We used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposure-specific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard). Results: We discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders. Conclusions: Outcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed.http://www.sciencedirect.com/science/article/pii/S0160412018327302 |
| spellingShingle | Elizabeth G. Radke Barbara Glenn Audrey Galizia Amanda Persad Rebecca Nachman Thomas Bateson J. Michael Wright Ana Navas-Acien Whitney D. Arroyave Robin C. Puett Emily W. Harville Anna Z. Pollack Jane S. Burns Courtney D. Lynch Sharon K. Sagiv Cheryl Stein Glinda S. Cooper Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies Environment International |
| title | Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies |
| title_full | Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies |
| title_fullStr | Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies |
| title_full_unstemmed | Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies |
| title_short | Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies |
| title_sort | development of outcome specific criteria for study evaluation in systematic reviews of epidemiology studies |
| url | http://www.sciencedirect.com/science/article/pii/S0160412018327302 |
| work_keys_str_mv | AT elizabethgradke developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT barbaraglenn developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT audreygalizia developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT amandapersad developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT rebeccanachman developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT thomasbateson developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT jmichaelwright developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT ananavasacien developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT whitneydarroyave developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT robincpuett developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT emilywharville developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT annazpollack developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT janesburns developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT courtneydlynch developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT sharonksagiv developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT cherylstein developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies AT glindascooper developmentofoutcomespecificcriteriaforstudyevaluationinsystematicreviewsofepidemiologystudies |