The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development
Background: Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Sc...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2021-02-01
|
| Series: | Implementation Research and Practice |
| Online Access: | https://doi.org/10.1177/2633489521992553 |
| _version_ | 1819148373494595584 |
|---|---|
| author | Emily M Becker-Haimes Melanie R Klein Bryce D McLeod Sonja K Schoenwald Shannon Dorsey Aaron Hogue Perrin B Fugo Mary L Phan Carlin Hoffacker Rinad S Beidas |
| author_facet | Emily M Becker-Haimes Melanie R Klein Bryce D McLeod Sonja K Schoenwald Shannon Dorsey Aaron Hogue Perrin B Fugo Mary L Phan Carlin Hoffacker Rinad S Beidas |
| author_sort | Emily M Becker-Haimes |
| collection | DOAJ |
| description | Background: Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery. Approach: We describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians ( n = 6) and consultation with CBT experts ( n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising. Conclusion: Our community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems. Plain language summary Developing brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such “pragmatic” measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts. |
| first_indexed | 2024-12-22T13:44:41Z |
| format | Article |
| id | doaj.art-1051baee12af4b9b94a9ac743e5635d4 |
| institution | Directory Open Access Journal |
| issn | 2633-4895 |
| language | English |
| last_indexed | 2024-12-22T13:44:41Z |
| publishDate | 2021-02-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Implementation Research and Practice |
| spelling | doaj.art-1051baee12af4b9b94a9ac743e5635d42022-12-21T18:23:50ZengSAGE PublishingImplementation Research and Practice2633-48952021-02-01210.1177/2633489521992553The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure developmentEmily M Becker-Haimes0Melanie R Klein1Bryce D McLeod2Sonja K Schoenwald3Shannon Dorsey4Aaron Hogue5Perrin B Fugo6Mary L Phan7Carlin Hoffacker8Rinad S Beidas9Hall Mercer Community Mental Health, University of Pennsylvania Health System, Philadelphia, PA, USADepartment of Psychiatry, University of Pennsylvania, Philadelphia, PA, USADepartment of Psychology, Virginia Commonwealth University, Richmond, VA, USAOregon Social Learning Center, Eugene, OR, USADepartment of Psychology, University of Washington, Seattle, WA, USAPartnership to End Addiction, New York, NY, USADepartment of Psychiatry, University of Pennsylvania, Philadelphia, PA, USADepartment of Psychiatry, University of Pennsylvania, Philadelphia, PA, USADepartment of Psychiatry, University of Pennsylvania, Philadelphia, PA, USAPenn Implementation Science Center (PISCE@LDI), Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USABackground: Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery. Approach: We describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians ( n = 6) and consultation with CBT experts ( n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising. Conclusion: Our community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems. Plain language summary Developing brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such “pragmatic” measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts.https://doi.org/10.1177/2633489521992553 |
| spellingShingle | Emily M Becker-Haimes Melanie R Klein Bryce D McLeod Sonja K Schoenwald Shannon Dorsey Aaron Hogue Perrin B Fugo Mary L Phan Carlin Hoffacker Rinad S Beidas The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development Implementation Research and Practice |
| title | The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development |
| title_full | The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development |
| title_fullStr | The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development |
| title_full_unstemmed | The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development |
| title_short | The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development |
| title_sort | tpocs self reported therapist intervention fidelity for youth tpocs sertify a case study of pragmatic measure development |
| url | https://doi.org/10.1177/2633489521992553 |
| work_keys_str_mv | AT emilymbeckerhaimes thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT melanierklein thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT brycedmcleod thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT sonjakschoenwald thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT shannondorsey thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT aaronhogue thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT perrinbfugo thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT marylphan thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT carlinhoffacker thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT rinadsbeidas thetpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT emilymbeckerhaimes tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT melanierklein tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT brycedmcleod tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT sonjakschoenwald tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT shannondorsey tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT aaronhogue tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT perrinbfugo tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT marylphan tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT carlinhoffacker tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment AT rinadsbeidas tpocsselfreportedtherapistinterventionfidelityforyouthtpocssertifyacasestudyofpragmaticmeasuredevelopment |