Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study
Background With the elimination in 2021 of the United States Medical Licensing Examination Step 2 Clinical Skills test, it is incumbent upon U.S. medical schools to develop local validated assessments of clinical reasoning. While much attention has been paid to summative exams for graduating student...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-07-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-023-04424-9 |
| _version_ | 1827899885241761792 |
|---|---|
| author | Benjamin D. Gallagher Michael L. Green Jaideep S. Talwalkar |
| author_facet | Benjamin D. Gallagher Michael L. Green Jaideep S. Talwalkar |
| author_sort | Benjamin D. Gallagher |
| collection | DOAJ |
| description | Background With the elimination in 2021 of the United States Medical Licensing Examination Step 2 Clinical Skills test, it is incumbent upon U.S. medical schools to develop local validated assessments of clinical reasoning. While much attention has been paid to summative exams for graduating students, formative exams for pre-clerkship students have not been well studied. Methods We applied the University of Illinois at Chicago College of Medicine (UIC-COM) Patient Note (PN) Scoring Rubric to templated PNs written by 103 pre-clerkship students for two cases in an objective structured clinical examination (OSCE) at the Yale School of Medicine. The rubric consists of four section scores (Documentation, Differential Diagnosis, Justification, and Workup, each scored 1 to 4) and a composite score (scaled 23 to 100). We calculated item discrimination for each section score and Cronbach’s alpha for each case. We surveyed students about their experience writing the templated PN. Results Mean Documentation, Differential Diagnosis, Justification, Workup, and composite scores for case A were 2.16, 1.80, 1.65, 2.29, and 47.67, respectively. For case B, the scores were 2.13, 1.21, 1.60, 1.67, and 40.54, respectively. Item discrimination ranged from 0.41 to 0.80. Cronbach’s alpha for cases A and B was 0.48 and 0.25, respectively. A majority of the students felt that the exercise was useful and appropriate to their level of training. Conclusions Despite performing poorly, pre-clerkship students found the note-writing task beneficial. Reliability of the scoring rubric was suboptimal, and modifications are needed to make this exercise a suitable measure of clinical reasoning. |
| first_indexed | 2024-03-12T23:23:44Z |
| format | Article |
| id | doaj.art-db5fd8c3ebf9423489028f39f437c604 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-03-12T23:23:44Z |
| publishDate | 2023-07-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-db5fd8c3ebf9423489028f39f437c6042023-07-16T11:19:57ZengBMCBMC Medical Education1472-69202023-07-012311710.1186/s12909-023-04424-9Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot studyBenjamin D. Gallagher0Michael L. Green1Jaideep S. Talwalkar2Department of Internal Medicine, Yale School of MedicineDepartment of Internal Medicine, Yale School of MedicineDepartment of Internal Medicine, Yale School of MedicineBackground With the elimination in 2021 of the United States Medical Licensing Examination Step 2 Clinical Skills test, it is incumbent upon U.S. medical schools to develop local validated assessments of clinical reasoning. While much attention has been paid to summative exams for graduating students, formative exams for pre-clerkship students have not been well studied. Methods We applied the University of Illinois at Chicago College of Medicine (UIC-COM) Patient Note (PN) Scoring Rubric to templated PNs written by 103 pre-clerkship students for two cases in an objective structured clinical examination (OSCE) at the Yale School of Medicine. The rubric consists of four section scores (Documentation, Differential Diagnosis, Justification, and Workup, each scored 1 to 4) and a composite score (scaled 23 to 100). We calculated item discrimination for each section score and Cronbach’s alpha for each case. We surveyed students about their experience writing the templated PN. Results Mean Documentation, Differential Diagnosis, Justification, Workup, and composite scores for case A were 2.16, 1.80, 1.65, 2.29, and 47.67, respectively. For case B, the scores were 2.13, 1.21, 1.60, 1.67, and 40.54, respectively. Item discrimination ranged from 0.41 to 0.80. Cronbach’s alpha for cases A and B was 0.48 and 0.25, respectively. A majority of the students felt that the exercise was useful and appropriate to their level of training. Conclusions Despite performing poorly, pre-clerkship students found the note-writing task beneficial. Reliability of the scoring rubric was suboptimal, and modifications are needed to make this exercise a suitable measure of clinical reasoning.https://doi.org/10.1186/s12909-023-04424-9Undergraduate medical educationClinical reasoningClinical assessmentOSCEScoring rubric |
| spellingShingle | Benjamin D. Gallagher Michael L. Green Jaideep S. Talwalkar Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study BMC Medical Education Undergraduate medical education Clinical reasoning Clinical assessment OSCE Scoring rubric |
| title | Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study |
| title_full | Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study |
| title_fullStr | Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study |
| title_full_unstemmed | Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study |
| title_short | Applying a validated scoring rubric to pre-clerkship medical students’ standardized patient notes: a pilot study |
| title_sort | applying a validated scoring rubric to pre clerkship medical students standardized patient notes a pilot study |
| topic | Undergraduate medical education Clinical reasoning Clinical assessment OSCE Scoring rubric |
| url | https://doi.org/10.1186/s12909-023-04424-9 |
| work_keys_str_mv | AT benjamindgallagher applyingavalidatedscoringrubrictopreclerkshipmedicalstudentsstandardizedpatientnotesapilotstudy AT michaellgreen applyingavalidatedscoringrubrictopreclerkshipmedicalstudentsstandardizedpatientnotesapilotstudy AT jaideepstalwalkar applyingavalidatedscoringrubrictopreclerkshipmedicalstudentsstandardizedpatientnotesapilotstudy |