Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine
Abstract Background The most effective method of teaching critical appraisal concepts remains unclear. We used simulation scenarios in a Risk-of-Bias (RoB) 2.0 framework to teach the various biases that may affect randomized controlled trials and assessed whether including this interactive session i...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-10-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-023-04738-8 |
| _version_ | 1797558967974494208 |
|---|---|
| author | Ashleigh Peng Lin Yun-Yun Chou Ka-Wai Tam |
| author_facet | Ashleigh Peng Lin Yun-Yun Chou Ka-Wai Tam |
| author_sort | Ashleigh Peng Lin |
| collection | DOAJ |
| description | Abstract Background The most effective method of teaching critical appraisal concepts remains unclear. We used simulation scenarios in a Risk-of-Bias (RoB) 2.0 framework to teach the various biases that may affect randomized controlled trials and assessed whether including this interactive session in an evidence-based medicine (EBM) course for third-year preclinical medical students can optimize their understanding of critical appraisal concepts. Methods The session had 13 modules, each corresponding to a particular risk of bias in RoB 2.0. Each module included a simulated scenario, followed by data presentation and a generalized conclusion. The students were subsequently asked to use colored vote cards to indicate whether they agreed, had some concern, or disagreed with the conclusion and to justify their answers. On the basis of the students’ answers, the facilitator debriefed the scenario and addressed the specific bias. In each module, the students were required to demonstrate critical thinking in analyzing the claims and quality of the supporting evidence and in justifying their decisions, thus conceptualizing their understanding of research biases. Results We included 306 students across two pilot sessions in spring 2020 and 2021, and the response rate was 97.4%. The students were least able to discern the following problems: baseline imbalances when assessing allocation bias (correct answers: 9.06%), missing outcome data when assessing attrition bias (correct answers: 11.65%), and balanced nonprotocol interventions when assessing performance bias (correct answers: 14.88%). The postcourse survey revealed several aspects of the interactive session that the students appreciated or found challenging. Conclusion Preclinical medical students generally appreciated the inclusion of simulation scenarios and vote cards in an EBM course. The use of vote cards facilitated medical students’ understanding of critical appraisal concepts, uncovered areas that they found challenging to understand, and encouraged their active participation. Such interactive sessions should be increasingly included in medical education. |
| first_indexed | 2024-03-10T17:38:54Z |
| format | Article |
| id | doaj.art-b84c9cccf2b94ece9a8d1f90d8cbe792 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-03-10T17:38:54Z |
| publishDate | 2023-10-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-b84c9cccf2b94ece9a8d1f90d8cbe7922023-11-20T09:47:10ZengBMCBMC Medical Education1472-69202023-10-012311710.1186/s12909-023-04738-8Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicineAshleigh Peng Lin0Yun-Yun Chou1Ka-Wai Tam2Department of Medical Education, Taipei Veterans General HospitalShared Decision Making Resource Center, Shuang Ho Hospital, Taipei Medical UniversityShared Decision Making Resource Center, Shuang Ho Hospital, Taipei Medical UniversityAbstract Background The most effective method of teaching critical appraisal concepts remains unclear. We used simulation scenarios in a Risk-of-Bias (RoB) 2.0 framework to teach the various biases that may affect randomized controlled trials and assessed whether including this interactive session in an evidence-based medicine (EBM) course for third-year preclinical medical students can optimize their understanding of critical appraisal concepts. Methods The session had 13 modules, each corresponding to a particular risk of bias in RoB 2.0. Each module included a simulated scenario, followed by data presentation and a generalized conclusion. The students were subsequently asked to use colored vote cards to indicate whether they agreed, had some concern, or disagreed with the conclusion and to justify their answers. On the basis of the students’ answers, the facilitator debriefed the scenario and addressed the specific bias. In each module, the students were required to demonstrate critical thinking in analyzing the claims and quality of the supporting evidence and in justifying their decisions, thus conceptualizing their understanding of research biases. Results We included 306 students across two pilot sessions in spring 2020 and 2021, and the response rate was 97.4%. The students were least able to discern the following problems: baseline imbalances when assessing allocation bias (correct answers: 9.06%), missing outcome data when assessing attrition bias (correct answers: 11.65%), and balanced nonprotocol interventions when assessing performance bias (correct answers: 14.88%). The postcourse survey revealed several aspects of the interactive session that the students appreciated or found challenging. Conclusion Preclinical medical students generally appreciated the inclusion of simulation scenarios and vote cards in an EBM course. The use of vote cards facilitated medical students’ understanding of critical appraisal concepts, uncovered areas that they found challenging to understand, and encouraged their active participation. Such interactive sessions should be increasingly included in medical education.https://doi.org/10.1186/s12909-023-04738-8Critical appraisalEvidence-based medicineRisk of biasSimulation scenariosUndergraduate medical educationVote card |
| spellingShingle | Ashleigh Peng Lin Yun-Yun Chou Ka-Wai Tam Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine BMC Medical Education Critical appraisal Evidence-based medicine Risk of bias Simulation scenarios Undergraduate medical education Vote card |
| title | Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine |
| title_full | Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine |
| title_fullStr | Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine |
| title_full_unstemmed | Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine |
| title_short | Use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence-based medicine |
| title_sort | use of simulation scenarios and vote cards in teaching critical appraisal concepts in evidence based medicine |
| topic | Critical appraisal Evidence-based medicine Risk of bias Simulation scenarios Undergraduate medical education Vote card |
| url | https://doi.org/10.1186/s12909-023-04738-8 |
| work_keys_str_mv | AT ashleighpenglin useofsimulationscenariosandvotecardsinteachingcriticalappraisalconceptsinevidencebasedmedicine AT yunyunchou useofsimulationscenariosandvotecardsinteachingcriticalappraisalconceptsinevidencebasedmedicine AT kawaitam useofsimulationscenariosandvotecardsinteachingcriticalappraisalconceptsinevidencebasedmedicine |