Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation
Abstract Background Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners’ competence and adherence to the evidence-based parenting...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-06-01
|
| Series: | BMC Health Services Research |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12913-023-09611-4 |
| _version_ | 1827928770715058176 |
|---|---|
| author | Margrét Sigmarsdóttir Melanie M. Domenech Rodríguez Abigail Gewirtz Laura Rains Jolle Tjaden Marion S. Forgatch |
| author_facet | Margrét Sigmarsdóttir Melanie M. Domenech Rodríguez Abigail Gewirtz Laura Rains Jolle Tjaden Marion S. Forgatch |
| author_sort | Margrét Sigmarsdóttir |
| collection | DOAJ |
| description | Abstract Background Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners’ competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners’ fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period. Methods Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor’s two fidelity teams and the fidelity teams in seven implementation sites. Results Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams. Conclusions This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention. |
| first_indexed | 2024-03-13T06:12:23Z |
| format | Article |
| id | doaj.art-4694246f5f994ab885b104f09fc26306 |
| institution | Directory Open Access Journal |
| issn | 1472-6963 |
| language | English |
| last_indexed | 2024-03-13T06:12:23Z |
| publishDate | 2023-06-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Health Services Research |
| spelling | doaj.art-4694246f5f994ab885b104f09fc263062023-06-11T11:09:11ZengBMCBMC Health Services Research1472-69632023-06-0123111210.1186/s12913-023-09611-4Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementationMargrét Sigmarsdóttir0Melanie M. Domenech Rodríguez1Abigail Gewirtz2Laura Rains3Jolle Tjaden4Marion S. Forgatch5School of Education, University of IcelandDepartment of Psychology, Utah State UniversityDepartment of Psychology, Arizona State UniversityImplementation Sciences International, Inc.PI ResearchImplementation Sciences International, Inc.Abstract Background Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners’ competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners’ fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period. Methods Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor’s two fidelity teams and the fidelity teams in seven implementation sites. Results Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams. Conclusions This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention.https://doi.org/10.1186/s12913-023-09611-4ImplementationFidelityEvidence-based parenting programGenerationPMTO |
| spellingShingle | Margrét Sigmarsdóttir Melanie M. Domenech Rodríguez Abigail Gewirtz Laura Rains Jolle Tjaden Marion S. Forgatch Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation BMC Health Services Research Implementation Fidelity Evidence-based parenting program GenerationPMTO |
| title | Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation |
| title_full | Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation |
| title_fullStr | Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation |
| title_full_unstemmed | Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation |
| title_short | Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation |
| title_sort | five year fidelity assessment of an evidence based parenting program generationpmto inter rater reliability following international implementation |
| topic | Implementation Fidelity Evidence-based parenting program GenerationPMTO |
| url | https://doi.org/10.1186/s12913-023-09611-4 |
| work_keys_str_mv | AT margretsigmarsdottir fiveyearfidelityassessmentofanevidencebasedparentingprogramgenerationpmtointerraterreliabilityfollowinginternationalimplementation AT melaniemdomenechrodriguez fiveyearfidelityassessmentofanevidencebasedparentingprogramgenerationpmtointerraterreliabilityfollowinginternationalimplementation AT abigailgewirtz fiveyearfidelityassessmentofanevidencebasedparentingprogramgenerationpmtointerraterreliabilityfollowinginternationalimplementation AT laurarains fiveyearfidelityassessmentofanevidencebasedparentingprogramgenerationpmtointerraterreliabilityfollowinginternationalimplementation AT jolletjaden fiveyearfidelityassessmentofanevidencebasedparentingprogramgenerationpmtointerraterreliabilityfollowinginternationalimplementation AT marionsforgatch fiveyearfidelityassessmentofanevidencebasedparentingprogramgenerationpmtointerraterreliabilityfollowinginternationalimplementation |