Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study
Abstract Background Determining whether complex rehabilitation interventions are delivered with fidelity is important. Implementation fidelity can differ between sites, therapists delivering interventions and, over time, threatening trial outcomes and increasing the risk of type II and III errors. T...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-07-01
|
| Series: | Pilot and Feasibility Studies |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40814-022-01111-2 |
| _version_ | 1818501635147563008 |
|---|---|
| author | Jain Anne Holmes Joanna Clare Fletcher-Smith Jose Antonio Merchán-Baeza Julie Phillips Kathryn Radford |
| author_facet | Jain Anne Holmes Joanna Clare Fletcher-Smith Jose Antonio Merchán-Baeza Julie Phillips Kathryn Radford |
| author_sort | Jain Anne Holmes |
| collection | DOAJ |
| description | Abstract Background Determining whether complex rehabilitation interventions are delivered with fidelity is important. Implementation fidelity can differ between sites, therapists delivering interventions and, over time, threatening trial outcomes and increasing the risk of type II and III errors. This study aimed to develop a method of assessing occupational therapists’ fidelity to deliver a complex, individually tailored vocational rehabilitation (VR) intervention to people with traumatic brain injury (TBI) and assess the feasibility of its use in a randomised controlled trial. Methods Using mixed methods and drawing on the intervention logic model, we developed data collection tools to measure fidelity to early specialist TBI VR (ESTVR). Fidelity was measured quantitatively using intervention case report forms (CRF), fidelity checklists and clinical records. Qualitative data from mentoring records, interviews with intervention therapists, participants with TBI, employers and NHS staff at trial sites explored moderators of implementation fidelity. The conceptual framework of implementation fidelity (CFIF) guided measurement and analysis of and factors affecting fidelity. Data were triangulated and benchmarked against an earlier cohort study. Results Fidelity to a complex individually tailored VR intervention could be measured. Overall, OTs delivered ESTVR with fidelity. Different fidelity measures answered different questions, offering unique insights into fidelity. Fidelity was best assessed using a fidelity checklist, intervention CRFs and clinical notes. The OT clinical notes and mentoring records were best at identifying fidelity moderating factors. Interviews added little insight into fidelity moderating factors over and above mentoring or clinical records. Data triangulation offered a comprehensive assessment of fidelity, highlighting limitations of measurement methods and learning for future trials but was resource intensive. Interviews, fidelity visits and analysing clinical notes were also resource intense. Comparing fidelity data to a benchmark and using CFIF as a framework for organising the fidelity assessment helped. Conclusions OTs delivered the VR intervention with fidelity. A fidelity checklist and benchmark plus mentoring may offer a practical and effective way of measuring fidelity and identifying fidelity moderating factors in trials of complex individually-tailored rehabilitation interventions. Mentoring provided real-time indicators of and reasons for fidelity deviations. These methods require further evaluation. Trial registration ISRCTN Registry, ISRCTN38581822 (Registered: 02/01/2014). |
| first_indexed | 2024-12-10T20:58:59Z |
| format | Article |
| id | doaj.art-be9bb47c70cd49dea0ce4b5094773103 |
| institution | Directory Open Access Journal |
| issn | 2055-5784 |
| language | English |
| last_indexed | 2024-12-10T20:58:59Z |
| publishDate | 2022-07-01 |
| publisher | BMC |
| record_format | Article |
| series | Pilot and Feasibility Studies |
| spelling | doaj.art-be9bb47c70cd49dea0ce4b50947731032022-12-22T01:33:53ZengBMCPilot and Feasibility Studies2055-57842022-07-018111310.1186/s40814-022-01111-2Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility studyJain Anne Holmes0Joanna Clare Fletcher-Smith1Jose Antonio Merchán-Baeza2Julie Phillips3Kathryn Radford4Faculty of Medicine and Health Sciences, Centre for Rehabilitation and Ageing Research, B Floor, Medical School, Queen’s Medical Centre, The University of NottinghamFaculty of Medicine and Health Sciences, Centre for Rehabilitation and Ageing Research, B Floor, Medical School, Queen’s Medical Centre, The University of NottinghamFaculty of Health Science and Welfare, University of Vic-Central University of Catalonia (UVIC-UCC)Faculty of Medicine and Health Sciences, Centre for Rehabilitation and Ageing Research, B Floor, Medical School, Queen’s Medical Centre, The University of NottinghamFaculty of Medicine and Health Sciences, Centre for Rehabilitation and Ageing Research, B Floor, Medical School, Queen’s Medical Centre, The University of NottinghamAbstract Background Determining whether complex rehabilitation interventions are delivered with fidelity is important. Implementation fidelity can differ between sites, therapists delivering interventions and, over time, threatening trial outcomes and increasing the risk of type II and III errors. This study aimed to develop a method of assessing occupational therapists’ fidelity to deliver a complex, individually tailored vocational rehabilitation (VR) intervention to people with traumatic brain injury (TBI) and assess the feasibility of its use in a randomised controlled trial. Methods Using mixed methods and drawing on the intervention logic model, we developed data collection tools to measure fidelity to early specialist TBI VR (ESTVR). Fidelity was measured quantitatively using intervention case report forms (CRF), fidelity checklists and clinical records. Qualitative data from mentoring records, interviews with intervention therapists, participants with TBI, employers and NHS staff at trial sites explored moderators of implementation fidelity. The conceptual framework of implementation fidelity (CFIF) guided measurement and analysis of and factors affecting fidelity. Data were triangulated and benchmarked against an earlier cohort study. Results Fidelity to a complex individually tailored VR intervention could be measured. Overall, OTs delivered ESTVR with fidelity. Different fidelity measures answered different questions, offering unique insights into fidelity. Fidelity was best assessed using a fidelity checklist, intervention CRFs and clinical notes. The OT clinical notes and mentoring records were best at identifying fidelity moderating factors. Interviews added little insight into fidelity moderating factors over and above mentoring or clinical records. Data triangulation offered a comprehensive assessment of fidelity, highlighting limitations of measurement methods and learning for future trials but was resource intensive. Interviews, fidelity visits and analysing clinical notes were also resource intense. Comparing fidelity data to a benchmark and using CFIF as a framework for organising the fidelity assessment helped. Conclusions OTs delivered the VR intervention with fidelity. A fidelity checklist and benchmark plus mentoring may offer a practical and effective way of measuring fidelity and identifying fidelity moderating factors in trials of complex individually-tailored rehabilitation interventions. Mentoring provided real-time indicators of and reasons for fidelity deviations. These methods require further evaluation. Trial registration ISRCTN Registry, ISRCTN38581822 (Registered: 02/01/2014).https://doi.org/10.1186/s40814-022-01111-2ImplementationComplex interventionFidelityAdherenceModerating factorsVocational rehabilitation |
| spellingShingle | Jain Anne Holmes Joanna Clare Fletcher-Smith Jose Antonio Merchán-Baeza Julie Phillips Kathryn Radford Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study Pilot and Feasibility Studies Implementation Complex intervention Fidelity Adherence Moderating factors Vocational rehabilitation |
| title | Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study |
| title_full | Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study |
| title_fullStr | Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study |
| title_full_unstemmed | Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study |
| title_short | Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study |
| title_sort | developing a method to assess fidelity to a complex vocational rehabilitation intervention in the fresh trial a feasibility study |
| topic | Implementation Complex intervention Fidelity Adherence Moderating factors Vocational rehabilitation |
| url | https://doi.org/10.1186/s40814-022-01111-2 |
| work_keys_str_mv | AT jainanneholmes developingamethodtoassessfidelitytoacomplexvocationalrehabilitationinterventioninthefreshtrialafeasibilitystudy AT joannaclarefletchersmith developingamethodtoassessfidelitytoacomplexvocationalrehabilitationinterventioninthefreshtrialafeasibilitystudy AT joseantoniomerchanbaeza developingamethodtoassessfidelitytoacomplexvocationalrehabilitationinterventioninthefreshtrialafeasibilitystudy AT juliephillips developingamethodtoassessfidelitytoacomplexvocationalrehabilitationinterventioninthefreshtrialafeasibilitystudy AT kathrynradford developingamethodtoassessfidelitytoacomplexvocationalrehabilitationinterventioninthefreshtrialafeasibilitystudy |