Enhancing the defensibility of examiners’ marks in high stake OSCEs
Abstract Background Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline gr...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2018-01-01
|
| Series: | BMC Medical Education |
| Online Access: | http://link.springer.com/article/10.1186/s12909-017-1112-z |
| _version_ | 1818063249668571136 |
|---|---|
| author | Boaz Shulruf Arvin Damodaran Phil Jones Sean Kennedy George Mangos Anthony J. O’Sullivan Joel Rhee Silas Taylor Gary Velan Peter Harris |
| author_facet | Boaz Shulruf Arvin Damodaran Phil Jones Sean Kennedy George Mangos Anthony J. O’Sullivan Joel Rhee Silas Taylor Gary Velan Peter Harris |
| author_sort | Boaz Shulruf |
| collection | DOAJ |
| description | Abstract Background Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. Methods The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student’s t-tests, utilising bootstrapping, were used to compare the OBM2 with ‘traditional’ results. Interclass correlations were used to estimate the association between the grade reclassification and all other grades in this examination. Results The correlations between scores for each station and pass/fail outcomes increased significantly after the mark reclassification, yet the reclassification did not significantly impact on students’ total scores. Examiners, students and program leaders expressed high levels of satisfaction and the Faculty’s Curriculum Development Committee has decided that the OBM2 will be used for all future clinical examinations. Implications of the OBM2 are discussed. Conclusions The OBM2 provides a feasible, defensible and acceptable solution for classification of borderline grades as either pass or fail. |
| first_indexed | 2024-12-10T14:17:06Z |
| format | Article |
| id | doaj.art-b57355c9865c4bad830cbd80264466b8 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-12-10T14:17:06Z |
| publishDate | 2018-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-b57355c9865c4bad830cbd80264466b82022-12-22T01:45:19ZengBMCBMC Medical Education1472-69202018-01-011811910.1186/s12909-017-1112-zEnhancing the defensibility of examiners’ marks in high stake OSCEsBoaz Shulruf0Arvin Damodaran1Phil Jones2Sean Kennedy3George Mangos4Anthony J. O’Sullivan5Joel Rhee6Silas Taylor7Gary Velan8Peter Harris9Faculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWFaculty of Medicine, UNSWAbstract Background Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. Methods The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student’s t-tests, utilising bootstrapping, were used to compare the OBM2 with ‘traditional’ results. Interclass correlations were used to estimate the association between the grade reclassification and all other grades in this examination. Results The correlations between scores for each station and pass/fail outcomes increased significantly after the mark reclassification, yet the reclassification did not significantly impact on students’ total scores. Examiners, students and program leaders expressed high levels of satisfaction and the Faculty’s Curriculum Development Committee has decided that the OBM2 will be used for all future clinical examinations. Implications of the OBM2 are discussed. Conclusions The OBM2 provides a feasible, defensible and acceptable solution for classification of borderline grades as either pass or fail.http://link.springer.com/article/10.1186/s12909-017-1112-z |
| spellingShingle | Boaz Shulruf Arvin Damodaran Phil Jones Sean Kennedy George Mangos Anthony J. O’Sullivan Joel Rhee Silas Taylor Gary Velan Peter Harris Enhancing the defensibility of examiners’ marks in high stake OSCEs BMC Medical Education |
| title | Enhancing the defensibility of examiners’ marks in high stake OSCEs |
| title_full | Enhancing the defensibility of examiners’ marks in high stake OSCEs |
| title_fullStr | Enhancing the defensibility of examiners’ marks in high stake OSCEs |
| title_full_unstemmed | Enhancing the defensibility of examiners’ marks in high stake OSCEs |
| title_short | Enhancing the defensibility of examiners’ marks in high stake OSCEs |
| title_sort | enhancing the defensibility of examiners marks in high stake osces |
| url | http://link.springer.com/article/10.1186/s12909-017-1112-z |
| work_keys_str_mv | AT boazshulruf enhancingthedefensibilityofexaminersmarksinhighstakeosces AT arvindamodaran enhancingthedefensibilityofexaminersmarksinhighstakeosces AT philjones enhancingthedefensibilityofexaminersmarksinhighstakeosces AT seankennedy enhancingthedefensibilityofexaminersmarksinhighstakeosces AT georgemangos enhancingthedefensibilityofexaminersmarksinhighstakeosces AT anthonyjosullivan enhancingthedefensibilityofexaminersmarksinhighstakeosces AT joelrhee enhancingthedefensibilityofexaminersmarksinhighstakeosces AT silastaylor enhancingthedefensibilityofexaminersmarksinhighstakeosces AT garyvelan enhancingthedefensibilityofexaminersmarksinhighstakeosces AT peterharris enhancingthedefensibilityofexaminersmarksinhighstakeosces |