Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review
Abstract Background Process evaluations are increasingly conducted within pragmatic randomised controlled trials (RCTs) of health services interventions and provide vital information to enhance understanding of RCT findings. However, issues pertaining to process evaluation in this specific context h...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2020-11-01
|
| Series: | Trials |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s13063-020-04762-9 |
| _version_ | 1828809263799074816 |
|---|---|
| author | Caroline French Hilary Pinnock Gordon Forbes Imogen Skene Stephanie J. C. Taylor |
| author_facet | Caroline French Hilary Pinnock Gordon Forbes Imogen Skene Stephanie J. C. Taylor |
| author_sort | Caroline French |
| collection | DOAJ |
| description | Abstract Background Process evaluations are increasingly conducted within pragmatic randomised controlled trials (RCTs) of health services interventions and provide vital information to enhance understanding of RCT findings. However, issues pertaining to process evaluation in this specific context have been little discussed. We aimed to describe the frequency, characteristics, labelling, value, practical conduct issues, and accessibility of published process evaluations within pragmatic RCTs in health services research. Methods We used a 2-phase systematic search process to (1) identify an index sample of journal articles reporting primary outcome results of pragmatic RCTs published in 2015 and then (2) identify all associated publications. We used an operational definition of process evaluation based on the Medical Research Council’s process evaluation framework to identify both process evaluations reported separately and process data reported in the trial results papers. We extracted and analysed quantitative and qualitative data to answer review objectives. Results From an index sample of 31 pragmatic RCTs, we identified 17 separate process evaluation studies. These had varied characteristics and only three were labelled ‘process evaluation’. Each of the 31 trial results papers also reported process data, with a median of five different process evaluation components per trial. Reported barriers and facilitators related to real-world collection of process data, recruitment of participants to process evaluations, and health services research regulations. We synthesised a wide range of reported benefits of process evaluations to interventions, trials, and wider knowledge. Visibility was often poor, with 13/17 process evaluations not mentioned in the trial results paper and 12/16 process evaluation journal articles not appearing in the trial registry. Conclusions In our sample of reviewed pragmatic RCTs, the meaning of the label ‘process evaluation’ appears uncertain, and the scope and significance of the term warrant further research and clarification. Although there were many ways in which the process evaluations added value, they often had poor visibility. Our findings suggest approaches that could enhance the planning and utility of process evaluations in the context of pragmatic RCTs. Trial registration Not applicable for PROSPERO registration |
| first_indexed | 2024-12-12T08:53:24Z |
| format | Article |
| id | doaj.art-72be0cf9c23f44af9fb18b779a770705 |
| institution | Directory Open Access Journal |
| issn | 1745-6215 |
| language | English |
| last_indexed | 2024-12-12T08:53:24Z |
| publishDate | 2020-11-01 |
| publisher | BMC |
| record_format | Article |
| series | Trials |
| spelling | doaj.art-72be0cf9c23f44af9fb18b779a7707052022-12-22T00:30:05ZengBMCTrials1745-62152020-11-0121111610.1186/s13063-020-04762-9Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic reviewCaroline French0Hilary Pinnock1Gordon Forbes2Imogen Skene3Stephanie J. C. Taylor4Institute of Population Health Sciences, Barts and the London School of Medicine and Dentistry, Queen Mary University of LondonUsher Institute, The University of EdinburghInstitute of Psychiatry, Psychology and Neuroscience (IoPPN), Kings College LondonEmergency Department, Royal London Hospital, Barts Health NHS TrustInstitute of Population Health Sciences, Barts and the London School of Medicine and Dentistry, Queen Mary University of LondonAbstract Background Process evaluations are increasingly conducted within pragmatic randomised controlled trials (RCTs) of health services interventions and provide vital information to enhance understanding of RCT findings. However, issues pertaining to process evaluation in this specific context have been little discussed. We aimed to describe the frequency, characteristics, labelling, value, practical conduct issues, and accessibility of published process evaluations within pragmatic RCTs in health services research. Methods We used a 2-phase systematic search process to (1) identify an index sample of journal articles reporting primary outcome results of pragmatic RCTs published in 2015 and then (2) identify all associated publications. We used an operational definition of process evaluation based on the Medical Research Council’s process evaluation framework to identify both process evaluations reported separately and process data reported in the trial results papers. We extracted and analysed quantitative and qualitative data to answer review objectives. Results From an index sample of 31 pragmatic RCTs, we identified 17 separate process evaluation studies. These had varied characteristics and only three were labelled ‘process evaluation’. Each of the 31 trial results papers also reported process data, with a median of five different process evaluation components per trial. Reported barriers and facilitators related to real-world collection of process data, recruitment of participants to process evaluations, and health services research regulations. We synthesised a wide range of reported benefits of process evaluations to interventions, trials, and wider knowledge. Visibility was often poor, with 13/17 process evaluations not mentioned in the trial results paper and 12/16 process evaluation journal articles not appearing in the trial registry. Conclusions In our sample of reviewed pragmatic RCTs, the meaning of the label ‘process evaluation’ appears uncertain, and the scope and significance of the term warrant further research and clarification. Although there were many ways in which the process evaluations added value, they often had poor visibility. Our findings suggest approaches that could enhance the planning and utility of process evaluations in the context of pragmatic RCTs. Trial registration Not applicable for PROSPERO registrationhttp://link.springer.com/article/10.1186/s13063-020-04762-9Process evaluationPragmatic randomised controlled trialsHealth services research |
| spellingShingle | Caroline French Hilary Pinnock Gordon Forbes Imogen Skene Stephanie J. C. Taylor Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review Trials Process evaluation Pragmatic randomised controlled trials Health services research |
| title | Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review |
| title_full | Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review |
| title_fullStr | Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review |
| title_full_unstemmed | Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review |
| title_short | Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?—a systematic review |
| title_sort | process evaluation within pragmatic randomised controlled trials what is it why is it done and can we find it a systematic review |
| topic | Process evaluation Pragmatic randomised controlled trials Health services research |
| url | http://link.springer.com/article/10.1186/s13063-020-04762-9 |
| work_keys_str_mv | AT carolinefrench processevaluationwithinpragmaticrandomisedcontrolledtrialswhatisitwhyisitdoneandcanwefinditasystematicreview AT hilarypinnock processevaluationwithinpragmaticrandomisedcontrolledtrialswhatisitwhyisitdoneandcanwefinditasystematicreview AT gordonforbes processevaluationwithinpragmaticrandomisedcontrolledtrialswhatisitwhyisitdoneandcanwefinditasystematicreview AT imogenskene processevaluationwithinpragmaticrandomisedcontrolledtrialswhatisitwhyisitdoneandcanwefinditasystematicreview AT stephaniejctaylor processevaluationwithinpragmaticrandomisedcontrolledtrialswhatisitwhyisitdoneandcanwefinditasystematicreview |