Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education
Abstract Background Medical students often struggle to understand the relevance of Evidence Based Medicine (EBM) to their clinical practice, yet it is a competence that all students must develop prior to graduation. Objective structured clinical examinations (OSCEs) are a valued assessment tool to a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2021-04-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-021-02650-7 |
| _version_ | 1819117976454955008 |
|---|---|
| author | B. Kumaravel C. Stewart D. Ilic |
| author_facet | B. Kumaravel C. Stewart D. Ilic |
| author_sort | B. Kumaravel |
| collection | DOAJ |
| description | Abstract Background Medical students often struggle to understand the relevance of Evidence Based Medicine (EBM) to their clinical practice, yet it is a competence that all students must develop prior to graduation. Objective structured clinical examinations (OSCEs) are a valued assessment tool to assess critical components of EBM competency, particularly different levels of mastery as they progress through the course. This study developed and evaluated EBM based OSCE stations with an aim to establish a spiral approach for EBM OSCE stations for undergraduate medical students. Methods OSCE stations were developed with increasingly complex EBM tasks. OSCE stations were classified according to the classification rubric for EBP assessment tools (CREATE) framework and mapped against the recently published core competencies for evidence-based practice (EBP). Performance data evaluation was undertaken using Classical Test Theory analysing mean scores, pass rates, and station item total correlation (ITC) using SPSS. Results Six EBM based OSCE stations assessing various stages of EBM were created for use in high stakes summative OSCEs for different year groups across the undergraduate medical degree. All OSCE stations, except for one, had excellent correlation coefficients and hence a high reliability, ranging from 0.21–0.49. The domain mean score ranged from 13.33 to 16.83 out of 20. High reliability was demonstrated for the each of the summative OSCE circuits (Cronbach’s alpha = 0.67–0.85). In the CREATE framework these stations assessed knowledge, skills, and behaviour of medical students in asking, searching, appraising, and integrating evidence in practice. The OSCE stations were useful in assessing six core evidence-based practice competencies, which are meant to be practiced with exercises. A spiral model of OSCEs of increasing complexity was proposed to assess EBM competency as students progressed through the MBChB course. Conclusions The use of the OSCEs is a feasible method of authentically assessing leaner EBM performance and behaviour in a high stakes assessment setting. Use of valid and reliable EBM-based OSCE stations provide evidence for continued development of a hierarchy of assessing scaffolded learning and mastery of EBM competency. Further work is needed to assess their predictive validity. |
| first_indexed | 2024-12-22T05:41:32Z |
| format | Article |
| id | doaj.art-1c269f106d20479585034fdf3374ae32 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-12-22T05:41:32Z |
| publishDate | 2021-04-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-1c269f106d20479585034fdf3374ae322022-12-21T18:37:11ZengBMCBMC Medical Education1472-69202021-04-012111910.1186/s12909-021-02650-7Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical educationB. Kumaravel0C. Stewart1D. Ilic2The University of Buckingham Medical SchoolUniversity of NottinghamSchool of Public Health and Preventive Medicine, Monash UniversityAbstract Background Medical students often struggle to understand the relevance of Evidence Based Medicine (EBM) to their clinical practice, yet it is a competence that all students must develop prior to graduation. Objective structured clinical examinations (OSCEs) are a valued assessment tool to assess critical components of EBM competency, particularly different levels of mastery as they progress through the course. This study developed and evaluated EBM based OSCE stations with an aim to establish a spiral approach for EBM OSCE stations for undergraduate medical students. Methods OSCE stations were developed with increasingly complex EBM tasks. OSCE stations were classified according to the classification rubric for EBP assessment tools (CREATE) framework and mapped against the recently published core competencies for evidence-based practice (EBP). Performance data evaluation was undertaken using Classical Test Theory analysing mean scores, pass rates, and station item total correlation (ITC) using SPSS. Results Six EBM based OSCE stations assessing various stages of EBM were created for use in high stakes summative OSCEs for different year groups across the undergraduate medical degree. All OSCE stations, except for one, had excellent correlation coefficients and hence a high reliability, ranging from 0.21–0.49. The domain mean score ranged from 13.33 to 16.83 out of 20. High reliability was demonstrated for the each of the summative OSCE circuits (Cronbach’s alpha = 0.67–0.85). In the CREATE framework these stations assessed knowledge, skills, and behaviour of medical students in asking, searching, appraising, and integrating evidence in practice. The OSCE stations were useful in assessing six core evidence-based practice competencies, which are meant to be practiced with exercises. A spiral model of OSCEs of increasing complexity was proposed to assess EBM competency as students progressed through the MBChB course. Conclusions The use of the OSCEs is a feasible method of authentically assessing leaner EBM performance and behaviour in a high stakes assessment setting. Use of valid and reliable EBM-based OSCE stations provide evidence for continued development of a hierarchy of assessing scaffolded learning and mastery of EBM competency. Further work is needed to assess their predictive validity.https://doi.org/10.1186/s12909-021-02650-7Evidence-based medicineUndergraduate medical curriculumCompetencyOSCEsSummative |
| spellingShingle | B. Kumaravel C. Stewart D. Ilic Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education BMC Medical Education Evidence-based medicine Undergraduate medical curriculum Competency OSCEs Summative |
| title | Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education |
| title_full | Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education |
| title_fullStr | Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education |
| title_full_unstemmed | Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education |
| title_short | Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education |
| title_sort | development and evaluation of a spiral model of assessing ebm competency using osces in undergraduate medical education |
| topic | Evidence-based medicine Undergraduate medical curriculum Competency OSCEs Summative |
| url | https://doi.org/10.1186/s12909-021-02650-7 |
| work_keys_str_mv | AT bkumaravel developmentandevaluationofaspiralmodelofassessingebmcompetencyusingoscesinundergraduatemedicaleducation AT cstewart developmentandevaluationofaspiralmodelofassessingebmcompetencyusingoscesinundergraduatemedicaleducation AT dilic developmentandevaluationofaspiralmodelofassessingebmcompetencyusingoscesinundergraduatemedicaleducation |