Enhancing review criteria for dissemination and implementation science grants
Abstract Background The existing grant review criteria do not consider unique methods and priorities of Dissemination and Implementation Science (DIS). The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) scoring system includes 10 criteria based on Proctor et al.’s “te...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-02-01
|
| Series: | Implementation Science Communications |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s43058-023-00399-2 |
| _version_ | 1797864543125241856 |
|---|---|
| author | Nicole A. Stadnick Clare Viglione Erika L. Crable Jessica L. Montoya Maryam Gholami Irene Su Borsika Rabin |
| author_facet | Nicole A. Stadnick Clare Viglione Erika L. Crable Jessica L. Montoya Maryam Gholami Irene Su Borsika Rabin |
| author_sort | Nicole A. Stadnick |
| collection | DOAJ |
| description | Abstract Background The existing grant review criteria do not consider unique methods and priorities of Dissemination and Implementation Science (DIS). The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) scoring system includes 10 criteria based on Proctor et al.’s “ten key ingredients” and was developed to support the assessment of DIS research proposals. We describe how we adapted INSPECT and used it in combination with the NIH scoring system to evaluate pilot DIS study proposals through our DIS Center. Methods We adapted INSPECT to broaden considerations for diverse DIS settings and concepts (e.g., explicitly including dissemination and implementation methods). Five PhD-level researchers with intermediate to advanced DIS knowledge were trained to conduct reviews of seven grant applications using both the INSPECT and NIH criteria. The INSPECT overall scores range from 0 to 30 (higher scores are better), and the NIH overall scores range from 1 to 9 (lower scores are better). Each grant was independently reviewed by two reviewers, then discussed in a group meeting to compare the experiences using both criteria to evaluate the proposal and to finalize scoring decisions. A follow-up survey was sent to grant reviewers to solicit further reflections on each scoring criterion. Results Averaged across reviewers, the INSPECT overall scores ranged from 13 to 24, while the NIH overall scores ranged from 2 to 5. Reviewer reflections highlighted the unique value and utility for each scoring criterion. The NIH criteria had a broad scientific purview and were better suited to evaluate more effectiveness-focused and pre-implementation proposals not testing implementation strategies. The INSPECT criteria were easier to rate in terms of the quality of integrating DIS considerations into the proposal and to assess the potential for generalizability, real-world feasibility, and impact. Overall, reviewers noted that INSPECT was a helpful tool to guide DIS research proposal writing. Conclusions We confirmed complementarity in using both scoring criteria in our pilot study grant proposal review and highlighted the utility of INSPECT as a potential DIS resource for training and capacity building. Possible refinements to INSPECT include more explicit reviewer guidance on assessing pre-implementation proposals, providing reviewers with the opportunity to submit written commentary with each numerical rating, and greater clarity on rating criteria with overlapping descriptions. |
| first_indexed | 2024-04-09T22:53:36Z |
| format | Article |
| id | doaj.art-1d3ba87788194501b7a83bf2ac0f74a8 |
| institution | Directory Open Access Journal |
| issn | 2662-2211 |
| language | English |
| last_indexed | 2024-04-09T22:53:36Z |
| publishDate | 2023-02-01 |
| publisher | BMC |
| record_format | Article |
| series | Implementation Science Communications |
| spelling | doaj.art-1d3ba87788194501b7a83bf2ac0f74a82023-03-22T11:21:52ZengBMCImplementation Science Communications2662-22112023-02-01411610.1186/s43058-023-00399-2Enhancing review criteria for dissemination and implementation science grantsNicole A. Stadnick0Clare Viglione1Erika L. Crable2Jessica L. Montoya3Maryam Gholami4Irene Su5Borsika Rabin6Dissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteDissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteDissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteDissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteDissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteDissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteDissemination and Implementation Science Center, UC San Diego Altman Clinical and Translational Research InstituteAbstract Background The existing grant review criteria do not consider unique methods and priorities of Dissemination and Implementation Science (DIS). The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) scoring system includes 10 criteria based on Proctor et al.’s “ten key ingredients” and was developed to support the assessment of DIS research proposals. We describe how we adapted INSPECT and used it in combination with the NIH scoring system to evaluate pilot DIS study proposals through our DIS Center. Methods We adapted INSPECT to broaden considerations for diverse DIS settings and concepts (e.g., explicitly including dissemination and implementation methods). Five PhD-level researchers with intermediate to advanced DIS knowledge were trained to conduct reviews of seven grant applications using both the INSPECT and NIH criteria. The INSPECT overall scores range from 0 to 30 (higher scores are better), and the NIH overall scores range from 1 to 9 (lower scores are better). Each grant was independently reviewed by two reviewers, then discussed in a group meeting to compare the experiences using both criteria to evaluate the proposal and to finalize scoring decisions. A follow-up survey was sent to grant reviewers to solicit further reflections on each scoring criterion. Results Averaged across reviewers, the INSPECT overall scores ranged from 13 to 24, while the NIH overall scores ranged from 2 to 5. Reviewer reflections highlighted the unique value and utility for each scoring criterion. The NIH criteria had a broad scientific purview and were better suited to evaluate more effectiveness-focused and pre-implementation proposals not testing implementation strategies. The INSPECT criteria were easier to rate in terms of the quality of integrating DIS considerations into the proposal and to assess the potential for generalizability, real-world feasibility, and impact. Overall, reviewers noted that INSPECT was a helpful tool to guide DIS research proposal writing. Conclusions We confirmed complementarity in using both scoring criteria in our pilot study grant proposal review and highlighted the utility of INSPECT as a potential DIS resource for training and capacity building. Possible refinements to INSPECT include more explicit reviewer guidance on assessing pre-implementation proposals, providing reviewers with the opportunity to submit written commentary with each numerical rating, and greater clarity on rating criteria with overlapping descriptions.https://doi.org/10.1186/s43058-023-00399-2Review criteriaGrantsResearch proposalsDissemination and implementation science |
| spellingShingle | Nicole A. Stadnick Clare Viglione Erika L. Crable Jessica L. Montoya Maryam Gholami Irene Su Borsika Rabin Enhancing review criteria for dissemination and implementation science grants Implementation Science Communications Review criteria Grants Research proposals Dissemination and implementation science |
| title | Enhancing review criteria for dissemination and implementation science grants |
| title_full | Enhancing review criteria for dissemination and implementation science grants |
| title_fullStr | Enhancing review criteria for dissemination and implementation science grants |
| title_full_unstemmed | Enhancing review criteria for dissemination and implementation science grants |
| title_short | Enhancing review criteria for dissemination and implementation science grants |
| title_sort | enhancing review criteria for dissemination and implementation science grants |
| topic | Review criteria Grants Research proposals Dissemination and implementation science |
| url | https://doi.org/10.1186/s43058-023-00399-2 |
| work_keys_str_mv | AT nicoleastadnick enhancingreviewcriteriafordisseminationandimplementationsciencegrants AT clareviglione enhancingreviewcriteriafordisseminationandimplementationsciencegrants AT erikalcrable enhancingreviewcriteriafordisseminationandimplementationsciencegrants AT jessicalmontoya enhancingreviewcriteriafordisseminationandimplementationsciencegrants AT maryamgholami enhancingreviewcriteriafordisseminationandimplementationsciencegrants AT irenesu enhancingreviewcriteriafordisseminationandimplementationsciencegrants AT borsikarabin enhancingreviewcriteriafordisseminationandimplementationsciencegrants |