Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England
IntroductionRapid evaluations can offer evidence on innovations in health and social care that can be used to inform fast-moving policy and practise, and support their scale-up according to previous research. However, there are few comprehensive accounts of how to plan and conduct large-scale rapid...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-02-01
|
| Series: | Frontiers in Sociology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fsoc.2023.982946/full |
| _version_ | 1811165072898129920 |
|---|---|
| author | Holly Walton Nadia E. Crellin Manbinder S. Sidhu Chris Sherlaw-Johnson Lauren Herlitz Lauren Herlitz Ian Litchfield Theo Georghiou Sonila M. Tomini Efthalia Massou Jo Ellins Jon Sussex Naomi J. Fulop |
| author_facet | Holly Walton Nadia E. Crellin Manbinder S. Sidhu Chris Sherlaw-Johnson Lauren Herlitz Lauren Herlitz Ian Litchfield Theo Georghiou Sonila M. Tomini Efthalia Massou Jo Ellins Jon Sussex Naomi J. Fulop |
| author_sort | Holly Walton |
| collection | DOAJ |
| description | IntroductionRapid evaluations can offer evidence on innovations in health and social care that can be used to inform fast-moving policy and practise, and support their scale-up according to previous research. However, there are few comprehensive accounts of how to plan and conduct large-scale rapid evaluations, ensure scientific rigour, and achieve stakeholder engagement within compressed timeframes.MethodsUsing a case study of a national mixed-methods rapid evaluation of COVID-19 remote home monitoring services in England, conducted during the COVID-19 pandemic, this manuscript examines the process of conducting a large-scale rapid evaluation from design to dissemination and impact, and reflects on the key lessons for conducting future large-scale rapid evaluations. In this manuscript, we describe each stage of the rapid evaluation: convening the team (study team and external collaborators), design and planning (scoping, designing protocols, study set up), data collection and analysis, and dissemination.ResultsWe reflect on why certain decisions were made and highlight facilitators and challenges. The manuscript concludes with 12 key lessons for conducting large-scale mixed-methods rapid evaluations of healthcare services. We propose that rapid study teams need to: (1) find ways of quickly building trust with external stakeholders, including evidence-users; (2) consider the needs of the rapid evaluation and resources needed; (3) use scoping to ensure the study is highly focused; (4) carefully consider what cannot be completed within a designated timeframe; (5) use structured processes to ensure consistency and rigour; (6) be flexible and responsive to changing needs and circumstances; (7) consider the risks associated with new data collection approaches of quantitative data (and their usability); (8) consider whether it is possible to use aggregated quantitative data, and what that would mean when presenting results, (9) consider using structured processes & layered analysis approaches to rapidly synthesise qualitative findings, (10) consider the balance between speed and the size and skills of the team, (11) ensure all team members know roles and responsibilities and can communicate quickly and clearly; and (12) consider how best to share findings, in discussion with evidence-users, for rapid understanding and use.ConclusionThese 12 lessons can be used to inform the development and conduct of future rapid evaluations in a range of contexts and settings. |
| first_indexed | 2024-04-10T15:31:36Z |
| format | Article |
| id | doaj.art-be10c164e266468ea45f5cdb10aaa60a |
| institution | Directory Open Access Journal |
| issn | 2297-7775 |
| language | English |
| last_indexed | 2024-04-10T15:31:36Z |
| publishDate | 2023-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Sociology |
| spelling | doaj.art-be10c164e266468ea45f5cdb10aaa60a2023-02-13T14:32:47ZengFrontiers Media S.A.Frontiers in Sociology2297-77752023-02-01810.3389/fsoc.2023.982946982946Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in EnglandHolly Walton0Nadia E. Crellin1Manbinder S. Sidhu2Chris Sherlaw-Johnson3Lauren Herlitz4Lauren Herlitz5Ian Litchfield6Theo Georghiou7Sonila M. Tomini8Efthalia Massou9Jo Ellins10Jon Sussex11Naomi J. Fulop12Department of Applied Health Research, University College London, London, United KingdomResearch and Policy, The Nuffield Trust, London, United KingdomHealth Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, United KingdomResearch and Policy, The Nuffield Trust, London, United KingdomDepartment of Applied Health Research, University College London, London, United KingdomPopulation, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, United KingdomInstitute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United KingdomResearch and Policy, The Nuffield Trust, London, United KingdomGlobal Business School for Health, University College London, London, United KingdomDepartment of Public Health and Primary Care, University of Cambridge, Cambridge, United KingdomHealth Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, United KingdomRAND Europe, Cambridge, United KingdomDepartment of Applied Health Research, University College London, London, United KingdomIntroductionRapid evaluations can offer evidence on innovations in health and social care that can be used to inform fast-moving policy and practise, and support their scale-up according to previous research. However, there are few comprehensive accounts of how to plan and conduct large-scale rapid evaluations, ensure scientific rigour, and achieve stakeholder engagement within compressed timeframes.MethodsUsing a case study of a national mixed-methods rapid evaluation of COVID-19 remote home monitoring services in England, conducted during the COVID-19 pandemic, this manuscript examines the process of conducting a large-scale rapid evaluation from design to dissemination and impact, and reflects on the key lessons for conducting future large-scale rapid evaluations. In this manuscript, we describe each stage of the rapid evaluation: convening the team (study team and external collaborators), design and planning (scoping, designing protocols, study set up), data collection and analysis, and dissemination.ResultsWe reflect on why certain decisions were made and highlight facilitators and challenges. The manuscript concludes with 12 key lessons for conducting large-scale mixed-methods rapid evaluations of healthcare services. We propose that rapid study teams need to: (1) find ways of quickly building trust with external stakeholders, including evidence-users; (2) consider the needs of the rapid evaluation and resources needed; (3) use scoping to ensure the study is highly focused; (4) carefully consider what cannot be completed within a designated timeframe; (5) use structured processes to ensure consistency and rigour; (6) be flexible and responsive to changing needs and circumstances; (7) consider the risks associated with new data collection approaches of quantitative data (and their usability); (8) consider whether it is possible to use aggregated quantitative data, and what that would mean when presenting results, (9) consider using structured processes & layered analysis approaches to rapidly synthesise qualitative findings, (10) consider the balance between speed and the size and skills of the team, (11) ensure all team members know roles and responsibilities and can communicate quickly and clearly; and (12) consider how best to share findings, in discussion with evidence-users, for rapid understanding and use.ConclusionThese 12 lessons can be used to inform the development and conduct of future rapid evaluations in a range of contexts and settings.https://www.frontiersin.org/articles/10.3389/fsoc.2023.982946/fullrapid evaluationreflectionskey lessonsCOVID-19mixed methods |
| spellingShingle | Holly Walton Nadia E. Crellin Manbinder S. Sidhu Chris Sherlaw-Johnson Lauren Herlitz Lauren Herlitz Ian Litchfield Theo Georghiou Sonila M. Tomini Efthalia Massou Jo Ellins Jon Sussex Naomi J. Fulop Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England Frontiers in Sociology rapid evaluation reflections key lessons COVID-19 mixed methods |
| title | Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England |
| title_full | Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England |
| title_fullStr | Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England |
| title_full_unstemmed | Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England |
| title_short | Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England |
| title_sort | undertaking rapid evaluations during the covid 19 pandemic lessons from evaluating covid 19 remote home monitoring services in england |
| topic | rapid evaluation reflections key lessons COVID-19 mixed methods |
| url | https://www.frontiersin.org/articles/10.3389/fsoc.2023.982946/full |
| work_keys_str_mv | AT hollywalton undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT nadiaecrellin undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT manbinderssidhu undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT chrissherlawjohnson undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT laurenherlitz undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT laurenherlitz undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT ianlitchfield undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT theogeorghiou undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT sonilamtomini undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT efthaliamassou undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT joellins undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT jonsussex undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland AT naomijfulop undertakingrapidevaluationsduringthecovid19pandemiclessonsfromevaluatingcovid19remotehomemonitoringservicesinengland |