Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial
BackgroundUS residents require practice and feedback to meet Accreditation Council for Graduate Medical Education mandates and patient expectations for effective communication after harmful errors. Current instructional approaches rely heavily on lectures, rarely provide indi...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2022-10-01
|
| Series: | JMIR Medical Education |
| Online Access: | https://mededu.jmir.org/2022/4/e40758 |
| _version_ | 1797734720338919424 |
|---|---|
| author | Andrew A White Ann M King Angelo E D’Addario Karen Berg Brigham Suzanne Dintzis Emily E Fay Thomas H Gallagher Kathleen M Mazor |
| author_facet | Andrew A White Ann M King Angelo E D’Addario Karen Berg Brigham Suzanne Dintzis Emily E Fay Thomas H Gallagher Kathleen M Mazor |
| author_sort | Andrew A White |
| collection | DOAJ |
| description |
BackgroundUS residents require practice and feedback to meet Accreditation Council for Graduate Medical Education mandates and patient expectations for effective communication after harmful errors. Current instructional approaches rely heavily on lectures, rarely provide individualized feedback to residents about communication skills, and may not assure that residents acquire the skills desired by patients. The Video-based Communication Assessment (VCA) app is a novel tool for simulating communication scenarios for practice and obtaining crowdsourced assessments and feedback on physicians’ communication skills. We previously established that crowdsourced laypeople can reliably assess residents’ error disclosure skills with the VCA app. However, its efficacy for error disclosure training has not been tested.
ObjectiveWe aimed to evaluate the efficacy of using VCA practice and feedback as a stand-alone intervention for the development of residents’ error disclosure skills.
MethodsWe conducted a pre-post study in 2020 with pathology, obstetrics and gynecology, and internal medicine residents at an academic medical center in the United States. At baseline, residents each completed 2 specialty-specific VCA cases depicting medical errors. Audio responses were rated by at least 8 crowdsourced laypeople using 6 items on a 5-point scale. At 4 weeks, residents received numerical and written feedback derived from layperson ratings and then completed 2 additional cases. Residents were randomly assigned cases at baseline and after feedback assessments to avoid ordinal effects. Ratings were aggregated to create overall assessment scores for each resident at baseline and after feedback. Residents completed a survey of demographic characteristics. We used a 2×3 split-plot ANOVA to test the effects of time (pre-post) and specialty on communication ratings.
ResultsIn total, 48 residents completed 2 cases at time 1, received a feedback report at 4 weeks, and completed 2 more cases. The mean ratings of residents’ communication were higher at time 2 versus time 1 (3.75 vs 3.53; P<.001). Residents with prior error disclosure experience performed better at time 1 compared to those without such experience (ratings: mean 3.63 vs mean 3.46; P=.02). No differences in communication ratings based on specialty or years in training were detected. Residents’ communication was rated higher for angry cases versus sad cases (mean 3.69 vs mean 3.58; P=.01). Less than half of all residents (27/62, 44%) reported prior experience with disclosing medical harm to patients; experience differed significantly among specialties (P<.001) and was lowest for pathology (1/17, 6%).
ConclusionsResidents at all training levels can potentially improve error disclosure skills with VCA practice and feedback. Error disclosure curricula should prepare residents for responding to various patient affects. Simulated error disclosure may particularly benefit trainees in diagnostic specialties, such as pathology, with infrequent real-life error disclosure practice opportunities. Future research should examine the effectiveness, feasibility, and acceptability of VCA within a longitudinal error disclosure curriculum. |
| first_indexed | 2024-03-12T12:47:26Z |
| format | Article |
| id | doaj.art-74e675a0859e4f9dba2d9e745e9f1c9c |
| institution | Directory Open Access Journal |
| issn | 2369-3762 |
| language | English |
| last_indexed | 2024-03-12T12:47:26Z |
| publishDate | 2022-10-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Education |
| spelling | doaj.art-74e675a0859e4f9dba2d9e745e9f1c9c2023-08-28T23:12:18ZengJMIR PublicationsJMIR Medical Education2369-37622022-10-0184e4075810.2196/40758Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post TrialAndrew A Whitehttps://orcid.org/0000-0002-9859-0947Ann M Kinghttps://orcid.org/0000-0002-6994-9711Angelo E D’Addariohttps://orcid.org/0000-0002-7687-3049Karen Berg Brighamhttps://orcid.org/0000-0002-3852-7591Suzanne Dintzishttps://orcid.org/0000-0001-9762-030XEmily E Fayhttps://orcid.org/0000-0001-5582-2835Thomas H Gallagherhttps://orcid.org/0000-0001-8323-1450Kathleen M Mazorhttps://orcid.org/0000-0002-9491-9872 BackgroundUS residents require practice and feedback to meet Accreditation Council for Graduate Medical Education mandates and patient expectations for effective communication after harmful errors. Current instructional approaches rely heavily on lectures, rarely provide individualized feedback to residents about communication skills, and may not assure that residents acquire the skills desired by patients. The Video-based Communication Assessment (VCA) app is a novel tool for simulating communication scenarios for practice and obtaining crowdsourced assessments and feedback on physicians’ communication skills. We previously established that crowdsourced laypeople can reliably assess residents’ error disclosure skills with the VCA app. However, its efficacy for error disclosure training has not been tested. ObjectiveWe aimed to evaluate the efficacy of using VCA practice and feedback as a stand-alone intervention for the development of residents’ error disclosure skills. MethodsWe conducted a pre-post study in 2020 with pathology, obstetrics and gynecology, and internal medicine residents at an academic medical center in the United States. At baseline, residents each completed 2 specialty-specific VCA cases depicting medical errors. Audio responses were rated by at least 8 crowdsourced laypeople using 6 items on a 5-point scale. At 4 weeks, residents received numerical and written feedback derived from layperson ratings and then completed 2 additional cases. Residents were randomly assigned cases at baseline and after feedback assessments to avoid ordinal effects. Ratings were aggregated to create overall assessment scores for each resident at baseline and after feedback. Residents completed a survey of demographic characteristics. We used a 2×3 split-plot ANOVA to test the effects of time (pre-post) and specialty on communication ratings. ResultsIn total, 48 residents completed 2 cases at time 1, received a feedback report at 4 weeks, and completed 2 more cases. The mean ratings of residents’ communication were higher at time 2 versus time 1 (3.75 vs 3.53; P<.001). Residents with prior error disclosure experience performed better at time 1 compared to those without such experience (ratings: mean 3.63 vs mean 3.46; P=.02). No differences in communication ratings based on specialty or years in training were detected. Residents’ communication was rated higher for angry cases versus sad cases (mean 3.69 vs mean 3.58; P=.01). Less than half of all residents (27/62, 44%) reported prior experience with disclosing medical harm to patients; experience differed significantly among specialties (P<.001) and was lowest for pathology (1/17, 6%). ConclusionsResidents at all training levels can potentially improve error disclosure skills with VCA practice and feedback. Error disclosure curricula should prepare residents for responding to various patient affects. Simulated error disclosure may particularly benefit trainees in diagnostic specialties, such as pathology, with infrequent real-life error disclosure practice opportunities. Future research should examine the effectiveness, feasibility, and acceptability of VCA within a longitudinal error disclosure curriculum.https://mededu.jmir.org/2022/4/e40758 |
| spellingShingle | Andrew A White Ann M King Angelo E D’Addario Karen Berg Brigham Suzanne Dintzis Emily E Fay Thomas H Gallagher Kathleen M Mazor Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial JMIR Medical Education |
| title | Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial |
| title_full | Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial |
| title_fullStr | Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial |
| title_full_unstemmed | Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial |
| title_short | Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians’ Adverse Event Communication Skills: Pre-post Trial |
| title_sort | effects of practicing with and obtaining crowdsourced feedback from the video based communication assessment app on resident physicians adverse event communication skills pre post trial |
| url | https://mededu.jmir.org/2022/4/e40758 |
| work_keys_str_mv | AT andrewawhite effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT annmking effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT angeloedaddario effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT karenbergbrigham effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT suzannedintzis effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT emilyefay effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT thomashgallagher effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial AT kathleenmmazor effectsofpracticingwithandobtainingcrowdsourcedfeedbackfromthevideobasedcommunicationassessmentapponresidentphysiciansadverseeventcommunicationskillspreposttrial |