The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs
Abstract Background Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-07-01
|
| Series: | Implementation Science |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13012-023-01285-0 |
| _version_ | 1797774033868029952 |
|---|---|
| author | Zoe M. Alley Jason E. Chapman Holle Schaper Lisa Saldana |
| author_facet | Zoe M. Alley Jason E. Chapman Holle Schaper Lisa Saldana |
| author_sort | Zoe M. Alley |
| collection | DOAJ |
| description | Abstract Background Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering services to clients (program start-up) and competency in treatment delivery. The present study evaluates the extent to which implementation process fidelity at different implementation stages predicts achievement of those milestones. Methods Implementation process fidelity data—as measured by the Stages of Implementation Completion (SIC)—from 1287 implementing sites across 27 evidence-informed programs were examined in mixed effects regression models with sites nested within programs. Implementation process fidelity, as measured by the proportion of implementation activities completed during the three stages of the SIC Pre-Implementation phase and overall Pre-Implementation (Phase 1) and Implementation (Phase 2) proportion scores, was assessed as a predictor of sites achieving program start-up (i.e., delivering services) and competency in program delivery. Results The predicted probability of start-up across all sites was low at 35% (95% CI [33%, 38%]). When considering the evidence-informed program being implemented, that probability was nearly twice as high (64%; 95% CI [42%, 82%]), and 57% of the total variance in program start-up was attributable to the program. Implementation process fidelity was positively and significantly associated with achievement of program start-up and competency. The magnitude of this relationship varied significantly across programs for Pre-Implementation Stage 1 (i.e., Engagement) only. Compared to other stages, completing more Pre-Implementation Stage 3 (Readiness Planning) activities resulted in the most rapid gains in probability of achieving program start-up. The predicted probability of achieving competency was very low unless sites had high scores in both Pre-Implementation and Implementation phases. Conclusions Strong implementation process fidelity—as measured by SIC Pre-Implementation and Implementation phase proportion scores—was associated with sites’ achievement of program start-up and competency in program delivery, with early implementation process fidelity being especially potent. These findings highlight the importance of a rigorous Pre-Implementation process. |
| first_indexed | 2024-03-12T22:15:05Z |
| format | Article |
| id | doaj.art-8e686aafee044959848c59ec77662168 |
| institution | Directory Open Access Journal |
| issn | 1748-5908 |
| language | English |
| last_indexed | 2024-03-12T22:15:05Z |
| publishDate | 2023-07-01 |
| publisher | BMC |
| record_format | Article |
| series | Implementation Science |
| spelling | doaj.art-8e686aafee044959848c59ec776621682023-07-23T11:21:28ZengBMCImplementation Science1748-59082023-07-0118111310.1186/s13012-023-01285-0The relative value of Pre-Implementation stages for successful implementation of evidence-informed programsZoe M. Alley0Jason E. Chapman1Holle Schaper2Lisa Saldana3Oregon Social Learning CenterOregon Social Learning CenterOregon Social Learning CenterOregon Social Learning CenterAbstract Background Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering services to clients (program start-up) and competency in treatment delivery. The present study evaluates the extent to which implementation process fidelity at different implementation stages predicts achievement of those milestones. Methods Implementation process fidelity data—as measured by the Stages of Implementation Completion (SIC)—from 1287 implementing sites across 27 evidence-informed programs were examined in mixed effects regression models with sites nested within programs. Implementation process fidelity, as measured by the proportion of implementation activities completed during the three stages of the SIC Pre-Implementation phase and overall Pre-Implementation (Phase 1) and Implementation (Phase 2) proportion scores, was assessed as a predictor of sites achieving program start-up (i.e., delivering services) and competency in program delivery. Results The predicted probability of start-up across all sites was low at 35% (95% CI [33%, 38%]). When considering the evidence-informed program being implemented, that probability was nearly twice as high (64%; 95% CI [42%, 82%]), and 57% of the total variance in program start-up was attributable to the program. Implementation process fidelity was positively and significantly associated with achievement of program start-up and competency. The magnitude of this relationship varied significantly across programs for Pre-Implementation Stage 1 (i.e., Engagement) only. Compared to other stages, completing more Pre-Implementation Stage 3 (Readiness Planning) activities resulted in the most rapid gains in probability of achieving program start-up. The predicted probability of achieving competency was very low unless sites had high scores in both Pre-Implementation and Implementation phases. Conclusions Strong implementation process fidelity—as measured by SIC Pre-Implementation and Implementation phase proportion scores—was associated with sites’ achievement of program start-up and competency in program delivery, with early implementation process fidelity being especially potent. These findings highlight the importance of a rigorous Pre-Implementation process.https://doi.org/10.1186/s13012-023-01285-0Implementation fidelityStages of implementation completionPre-Implementation |
| spellingShingle | Zoe M. Alley Jason E. Chapman Holle Schaper Lisa Saldana The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs Implementation Science Implementation fidelity Stages of implementation completion Pre-Implementation |
| title | The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs |
| title_full | The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs |
| title_fullStr | The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs |
| title_full_unstemmed | The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs |
| title_short | The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs |
| title_sort | relative value of pre implementation stages for successful implementation of evidence informed programs |
| topic | Implementation fidelity Stages of implementation completion Pre-Implementation |
| url | https://doi.org/10.1186/s13012-023-01285-0 |
| work_keys_str_mv | AT zoemalley therelativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT jasonechapman therelativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT holleschaper therelativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT lisasaldana therelativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT zoemalley relativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT jasonechapman relativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT holleschaper relativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms AT lisasaldana relativevalueofpreimplementationstagesforsuccessfulimplementationofevidenceinformedprograms |