Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment
Abstract Background Practical skill assessment is an important part of the learning process to confirm competencies in acquired medical knowledge. Objective This study aimed to compare the assessments of endotracheal intubation skills using the HybridLab® methodology between students and teacher in...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-04-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-023-04242-z |
| _version_ | 1827961260527845376 |
|---|---|
| author | Aida Mankute Laima Juozapaviciene Justinas Stucinskas Zilvinas Dambrauskas Paulius Dobozinskas Elizabeth Sinz David L. Rodgers Evelina Pukenyte Birute Kumpaitiene Dinas Vaitkaitis |
| author_facet | Aida Mankute Laima Juozapaviciene Justinas Stucinskas Zilvinas Dambrauskas Paulius Dobozinskas Elizabeth Sinz David L. Rodgers Evelina Pukenyte Birute Kumpaitiene Dinas Vaitkaitis |
| author_sort | Aida Mankute |
| collection | DOAJ |
| description | Abstract Background Practical skill assessment is an important part of the learning process to confirm competencies in acquired medical knowledge. Objective This study aimed to compare the assessments of endotracheal intubation skills using the HybridLab® methodology between students and teacher in terms of interobserver reliability. Methods Reliability analysis was performed with observational data (data are reported according to STROBE guidelines). The study was conducted in two countries, the Lithuanian University of Health Science (LUHS) and Pennsylvania State University (PSU) in the US, between 1 January and 30 June 2020. A total of 92 students (60 from LUHS and 32 from PSU) were trained in endotracheal intubation using an algorithm-driven hybrid learning method. At the end of the training session, the participants had to complete the evaluation scenario, which was assessed by one of the students and evaluated remotely by a single teacher. The student assessment of the endotracheal intubation procedure was compared with the teacher’s assessment using correlation and estimation of the intraclass correlation coefficient. Results Overall, the medians of the student and teacher assessments were both 100% (0%). Spearman’s correlation coefficient between the student and teacher assessments was 0.879 (p = 0.001). The intraclass correlation coefficient used for interobserver variations between the students and teacher was 0.883 (95% confidence interval from 0.824 to 0.923). Conclusions The algorithm-driven hybrid learning method allows students to reliably assess endotracheal intubation skills to a level comparable with that of the teacher’s evaluation. This learning method has the potential to be a cost-effective and efficient way to provide high-quality education while also saving human resources. |
| first_indexed | 2024-04-09T16:22:43Z |
| format | Article |
| id | doaj.art-5191f1aaaded44af9822635fc9ece212 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-04-09T16:22:43Z |
| publishDate | 2023-04-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-5191f1aaaded44af9822635fc9ece2122023-04-23T11:20:54ZengBMCBMC Medical Education1472-69202023-04-012311910.1186/s12909-023-04242-zInterrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environmentAida Mankute0Laima Juozapaviciene1Justinas Stucinskas2Zilvinas Dambrauskas3Paulius Dobozinskas4Elizabeth Sinz5David L. Rodgers6Evelina Pukenyte7Birute Kumpaitiene8Dinas Vaitkaitis9Department of Emergency Medicine, Lithuanian University of Health SciencesDepartment ofAnaesthesiology, Lithuanian University of Health SciencesDepartment of Orthopaedics and Traumatology, Lithuanian University of Health SciencesDepartment of Surgery, Lithuanian University of Health SciencesDepartment of Disaster Medicine, Lithuanian University of Health SciencesDepartment of Anesthesiology and Perioperative Medicine, Medical Simulation Center, Penn State Health Milton S. Hershey Medical CenterInterprofessional Simulation Center, Indiana UniversityDepartment of Infectious Diseases, Lithuanian University of Health SciencesDepartment of Cardiothoracic and Vascular Surgery, Lithuanian University of Health SciencesDepartment of Disaster Medicine, Lithuanian University of Health SciencesAbstract Background Practical skill assessment is an important part of the learning process to confirm competencies in acquired medical knowledge. Objective This study aimed to compare the assessments of endotracheal intubation skills using the HybridLab® methodology between students and teacher in terms of interobserver reliability. Methods Reliability analysis was performed with observational data (data are reported according to STROBE guidelines). The study was conducted in two countries, the Lithuanian University of Health Science (LUHS) and Pennsylvania State University (PSU) in the US, between 1 January and 30 June 2020. A total of 92 students (60 from LUHS and 32 from PSU) were trained in endotracheal intubation using an algorithm-driven hybrid learning method. At the end of the training session, the participants had to complete the evaluation scenario, which was assessed by one of the students and evaluated remotely by a single teacher. The student assessment of the endotracheal intubation procedure was compared with the teacher’s assessment using correlation and estimation of the intraclass correlation coefficient. Results Overall, the medians of the student and teacher assessments were both 100% (0%). Spearman’s correlation coefficient between the student and teacher assessments was 0.879 (p = 0.001). The intraclass correlation coefficient used for interobserver variations between the students and teacher was 0.883 (95% confidence interval from 0.824 to 0.923). Conclusions The algorithm-driven hybrid learning method allows students to reliably assess endotracheal intubation skills to a level comparable with that of the teacher’s evaluation. This learning method has the potential to be a cost-effective and efficient way to provide high-quality education while also saving human resources.https://doi.org/10.1186/s12909-023-04242-zHybridLabSelf-directed learningSkill assessmentStudentsTeacher |
| spellingShingle | Aida Mankute Laima Juozapaviciene Justinas Stucinskas Zilvinas Dambrauskas Paulius Dobozinskas Elizabeth Sinz David L. Rodgers Evelina Pukenyte Birute Kumpaitiene Dinas Vaitkaitis Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment BMC Medical Education HybridLab Self-directed learning Skill assessment Students Teacher |
| title | Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment |
| title_full | Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment |
| title_fullStr | Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment |
| title_full_unstemmed | Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment |
| title_short | Interrater agreement between student and teacher assessments of endotracheal intubation skills in a self-directed simulation learning environment |
| title_sort | interrater agreement between student and teacher assessments of endotracheal intubation skills in a self directed simulation learning environment |
| topic | HybridLab Self-directed learning Skill assessment Students Teacher |
| url | https://doi.org/10.1186/s12909-023-04242-z |
| work_keys_str_mv | AT aidamankute interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT laimajuozapaviciene interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT justinasstucinskas interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT zilvinasdambrauskas interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT pauliusdobozinskas interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT elizabethsinz interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT davidlrodgers interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT evelinapukenyte interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT birutekumpaitiene interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment AT dinasvaitkaitis interrateragreementbetweenstudentandteacherassessmentsofendotrachealintubationskillsinaselfdirectedsimulationlearningenvironment |