The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial
Objective and Study Setting: Research efforts to identify factors that influence successful implementation are growing. This paper describes methods of defining and measuring outcomes of implementation success, using a cluster randomized controlled trial with 12 cancer services in Australia comparin...
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2019-10-01
|
| Series: | Frontiers in Public Health |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fpubh.2019.00293/full |
| _version_ | 1818109716845297664 |
|---|---|
| author | Heather L. Shepherd Heather L. Shepherd Liesbeth Geerligs Phyllis Butow Phyllis Butow Lindy Masya Joanne Shaw Melanie Price Haryana M. Dhillon Haryana M. Dhillon Thomas F. Hack Thomas F. Hack Afaf Girgis Tim Luckett Melanie Lovell Melanie Lovell Brian Kelly Philip Beale Peter Grimison Peter Grimison Tim Shaw Rosalie Viney Nicole M. Rankin |
| author_facet | Heather L. Shepherd Heather L. Shepherd Liesbeth Geerligs Phyllis Butow Phyllis Butow Lindy Masya Joanne Shaw Melanie Price Haryana M. Dhillon Haryana M. Dhillon Thomas F. Hack Thomas F. Hack Afaf Girgis Tim Luckett Melanie Lovell Melanie Lovell Brian Kelly Philip Beale Peter Grimison Peter Grimison Tim Shaw Rosalie Viney Nicole M. Rankin |
| author_sort | Heather L. Shepherd |
| collection | DOAJ |
| description | Objective and Study Setting: Research efforts to identify factors that influence successful implementation are growing. This paper describes methods of defining and measuring outcomes of implementation success, using a cluster randomized controlled trial with 12 cancer services in Australia comparing the effectiveness of implementation strategies to support adherence to the Australian Clinical Pathway for the Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients (ADAPT CP).Study Design and Methods: Using the StaRI guidelines, a process evaluation was planned to explore participant experience of the ADAPT CP, resources and implementation strategies according to the Implementation Outcomes Framework. This study focused on identifying measurable outcome criteria, prior to data collection for the trial, which is currently in progress.Principal Findings: We translated each implementation outcome into clearly defined and measurable criteria, noting whether each addressed the ADAPT CP, resources or implementation strategies, or a combination of the three. A consensus process defined measures for the primary outcome (adherence) and secondary (implementation) outcomes; this process included literature review, discussion and clear measurement parameters. Based on our experience, we present an approach that could be used as a guide for other researchers and clinicians seeking to define success in their work.Conclusions: Defining and operationalizing success in real-world implementation yields a range of methodological challenges and complexities that may be overcome by iterative review and engagement with end users. A clear understanding of how outcomes are defined and measured, based on a strong theoretical framework, is crucial to meaningful measurement and outcomes. The conceptual approach described in this article could be generalized for use in other studies.Trial Registration: The ADAPT Program to support the management of anxiety and depression in adult cancer patients: a cluster randomized trial to evaluate different implementation strategies of the Clinical Pathway for Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients was prospectively registered with the Australian New Zealand Clinical Trials Registry Registration Number: ACTRN12617000411347. |
| first_indexed | 2024-12-11T02:35:41Z |
| format | Article |
| id | doaj.art-63e383f30e7b4f1eb6e39066c3e3b34c |
| institution | Directory Open Access Journal |
| issn | 2296-2565 |
| language | English |
| last_indexed | 2024-12-11T02:35:41Z |
| publishDate | 2019-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Public Health |
| spelling | doaj.art-63e383f30e7b4f1eb6e39066c3e3b34c2022-12-22T01:23:43ZengFrontiers Media S.A.Frontiers in Public Health2296-25652019-10-01710.3389/fpubh.2019.00293486602The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital TrialHeather L. Shepherd0Heather L. Shepherd1Liesbeth Geerligs2Phyllis Butow3Phyllis Butow4Lindy Masya5Joanne Shaw6Melanie Price7Haryana M. Dhillon8Haryana M. Dhillon9Thomas F. Hack10Thomas F. Hack11Afaf Girgis12Tim Luckett13Melanie Lovell14Melanie Lovell15Brian Kelly16Philip Beale17Peter Grimison18Peter Grimison19Tim Shaw20Rosalie Viney21Nicole M. Rankin22Psycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaCentre for Medical Psychology and Evidence-Based Decision-Making (CeMPED), University of Sydney, Sydney, NSW, AustraliaPsycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaPsycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaCentre for Medical Psychology and Evidence-Based Decision-Making (CeMPED), University of Sydney, Sydney, NSW, AustraliaPsycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaPsycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaPsycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaPsycho-Oncology Co-operative Research Group (PoCoG), University of Sydney, Sydney, NSW, AustraliaCentre for Medical Psychology and Evidence-Based Decision-Making (CeMPED), University of Sydney, Sydney, NSW, AustraliaRady Faculty of Health Sciences, College of Nursing, University of Manitoba, Winnipeg, MB, CanadaPsychosocial Oncology & Cancer Nursing Research, St. Boniface Hospital Research Centre, Winnipeg, MB, CanadaCentre for Oncology Education and Research Translation (CONCERT), Ingham Institute for Applied Medical Research, South Western Sydney Clinical School, University of New South Wales, Sydney, NSW, AustraliaImproving Palliative, Aged and Chronic Care Through Clinical Research and Translation (IMPACCT), Faculty of Health, University of Technology Sydney, Ultimo, NSW, AustraliaHammondCare Northern Sydney, Greenwich, NSW, AustraliaNorthern Clinical School, University of Sydney, Sydney, NSW, AustraliaSchool of Medicine and Public Health, University of Newcastle, Newcastle, NSW, Australia0Cancer Services for the Sydney Local Health District (Incorporating Royal Prince Alfred, Concord and Canterbury Hospitals), Sydney, NSW, Australia1Chris O'Brien Lifehouse, Camperdown, NSW, Australia2Sydney Medical School, University of Sydney, Sydney, NSW, Australia3Charles Perkins Centre, Faculty of Health Sciences, University of Sydney, Sydney, NSW, Australia4Centre for Health Economics Research and Evaluation, UTS Business School, University of Technology Sydney, Sydney, NSW, Australia5Faculty of Health Sciences, University of Sydney, Sydney, NSW, AustraliaObjective and Study Setting: Research efforts to identify factors that influence successful implementation are growing. This paper describes methods of defining and measuring outcomes of implementation success, using a cluster randomized controlled trial with 12 cancer services in Australia comparing the effectiveness of implementation strategies to support adherence to the Australian Clinical Pathway for the Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients (ADAPT CP).Study Design and Methods: Using the StaRI guidelines, a process evaluation was planned to explore participant experience of the ADAPT CP, resources and implementation strategies according to the Implementation Outcomes Framework. This study focused on identifying measurable outcome criteria, prior to data collection for the trial, which is currently in progress.Principal Findings: We translated each implementation outcome into clearly defined and measurable criteria, noting whether each addressed the ADAPT CP, resources or implementation strategies, or a combination of the three. A consensus process defined measures for the primary outcome (adherence) and secondary (implementation) outcomes; this process included literature review, discussion and clear measurement parameters. Based on our experience, we present an approach that could be used as a guide for other researchers and clinicians seeking to define success in their work.Conclusions: Defining and operationalizing success in real-world implementation yields a range of methodological challenges and complexities that may be overcome by iterative review and engagement with end users. A clear understanding of how outcomes are defined and measured, based on a strong theoretical framework, is crucial to meaningful measurement and outcomes. The conceptual approach described in this article could be generalized for use in other studies.Trial Registration: The ADAPT Program to support the management of anxiety and depression in adult cancer patients: a cluster randomized trial to evaluate different implementation strategies of the Clinical Pathway for Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients was prospectively registered with the Australian New Zealand Clinical Trials Registry Registration Number: ACTRN12617000411347.https://www.frontiersin.org/article/10.3389/fpubh.2019.00293/fullimplementation sciencehealth services researchoutcome measurementmethodologypsycho-oncologyclinical pathways |
| spellingShingle | Heather L. Shepherd Heather L. Shepherd Liesbeth Geerligs Phyllis Butow Phyllis Butow Lindy Masya Joanne Shaw Melanie Price Haryana M. Dhillon Haryana M. Dhillon Thomas F. Hack Thomas F. Hack Afaf Girgis Tim Luckett Melanie Lovell Melanie Lovell Brian Kelly Philip Beale Peter Grimison Peter Grimison Tim Shaw Rosalie Viney Nicole M. Rankin The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial Frontiers in Public Health implementation science health services research outcome measurement methodology psycho-oncology clinical pathways |
| title | The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial |
| title_full | The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial |
| title_fullStr | The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial |
| title_full_unstemmed | The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial |
| title_short | The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial |
| title_sort | elusive search for success defining and measuring implementation outcomes in a real world hospital trial |
| topic | implementation science health services research outcome measurement methodology psycho-oncology clinical pathways |
| url | https://www.frontiersin.org/article/10.3389/fpubh.2019.00293/full |
| work_keys_str_mv | AT heatherlshepherd theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT heatherlshepherd theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT liesbethgeerligs theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT phyllisbutow theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT phyllisbutow theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT lindymasya theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT joanneshaw theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT melanieprice theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT haryanamdhillon theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT haryanamdhillon theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT thomasfhack theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT thomasfhack theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT afafgirgis theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT timluckett theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT melanielovell theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT melanielovell theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT briankelly theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT philipbeale theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT petergrimison theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT petergrimison theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT timshaw theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT rosalieviney theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT nicolemrankin theelusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT heatherlshepherd elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT heatherlshepherd elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT liesbethgeerligs elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT phyllisbutow elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT phyllisbutow elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT lindymasya elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT joanneshaw elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT melanieprice elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT haryanamdhillon elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT haryanamdhillon elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT thomasfhack elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT thomasfhack elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT afafgirgis elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT timluckett elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT melanielovell elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT melanielovell elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT briankelly elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT philipbeale elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT petergrimison elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT petergrimison elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT timshaw elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT rosalieviney elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial AT nicolemrankin elusivesearchforsuccessdefiningandmeasuringimplementationoutcomesinarealworldhospitaltrial |