Simulation-based assessment of upper abdominal ultrasound skills
Abstract Background Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasoun...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMC
2024-01-01
|
Series: | BMC Medical Education |
Subjects: | |
Online Access: | https://doi.org/10.1186/s12909-023-05018-1 |
_version_ | 1797363180775669760 |
---|---|
author | Kristina E. Teslak Julie H. Post Martin G. Tolsgaard Sten Rasmussen Mathias M. Purup Mikkel L. Friis |
author_facet | Kristina E. Teslak Julie H. Post Martin G. Tolsgaard Sten Rasmussen Mathias M. Purup Mikkel L. Friis |
author_sort | Kristina E. Teslak |
collection | DOAJ |
description | Abstract Background Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasound skills using an ultrasound simulator. Methods Twenty five novices and five experts were recruited, all of whom completed an assessment program for the evaluation of upper abdominal ultrasound skills on a virtual reality simulator. The program included five modules that assessed different organ systems using automated simulator metrics. We used Messick’s framework to explore the validity evidence of these simulator metrics to determine the contents of a final simulator test. We used the contrasting groups method to establish a pass/fail level for the final simulator test. Results Thirty seven out of 60 metrics were able to discriminate between novices and experts (p < 0.05). The median simulator score of the final simulator test including the metrics with validity evidence was 26.68% (range: 8.1–40.5%) for novices and 85.1% (range: 56.8–91.9%) for experts. The internal structure was assessed by Cronbach alpha (0.93) and intraclass correlation coefficient (0.89). The pass/fail level was determined to be 50.9%. This pass/fail criterion found no passing novices or failing experts. Conclusions This study collected validity evidence for simulation-based assessment of upper abdominal ultrasound examinations, which is the first step toward competency-based training. Future studies may examine how competency-based training in the simulated setting translates into improvements in clinical performances. |
first_indexed | 2024-03-08T16:17:22Z |
format | Article |
id | doaj.art-7070e5c0c36f4bf093fd3f7e505b35da |
institution | Directory Open Access Journal |
issn | 1472-6920 |
language | English |
last_indexed | 2024-03-08T16:17:22Z |
publishDate | 2024-01-01 |
publisher | BMC |
record_format | Article |
series | BMC Medical Education |
spelling | doaj.art-7070e5c0c36f4bf093fd3f7e505b35da2024-01-07T12:31:10ZengBMCBMC Medical Education1472-69202024-01-012411710.1186/s12909-023-05018-1Simulation-based assessment of upper abdominal ultrasound skillsKristina E. Teslak0Julie H. Post1Martin G. Tolsgaard2Sten Rasmussen3Mathias M. Purup4Mikkel L. Friis5NordSim, Center for Skills Training and Simulation, Aalborg University HospitalNordSim, Center for Skills Training and Simulation, Aalborg University HospitalCopenhagen Academy for Medical Education and Simulation, RigshospitaletDepartment of Clinical Medicine, Aalborg UniversityDepartment of Radiology, Aalborg University HospitalNordSim, Center for Skills Training and Simulation, Aalborg University HospitalAbstract Background Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasound skills using an ultrasound simulator. Methods Twenty five novices and five experts were recruited, all of whom completed an assessment program for the evaluation of upper abdominal ultrasound skills on a virtual reality simulator. The program included five modules that assessed different organ systems using automated simulator metrics. We used Messick’s framework to explore the validity evidence of these simulator metrics to determine the contents of a final simulator test. We used the contrasting groups method to establish a pass/fail level for the final simulator test. Results Thirty seven out of 60 metrics were able to discriminate between novices and experts (p < 0.05). The median simulator score of the final simulator test including the metrics with validity evidence was 26.68% (range: 8.1–40.5%) for novices and 85.1% (range: 56.8–91.9%) for experts. The internal structure was assessed by Cronbach alpha (0.93) and intraclass correlation coefficient (0.89). The pass/fail level was determined to be 50.9%. This pass/fail criterion found no passing novices or failing experts. Conclusions This study collected validity evidence for simulation-based assessment of upper abdominal ultrasound examinations, which is the first step toward competency-based training. Future studies may examine how competency-based training in the simulated setting translates into improvements in clinical performances.https://doi.org/10.1186/s12909-023-05018-1Simulation-based ultrasound trainingValidity evidenceUltrasound assessment |
spellingShingle | Kristina E. Teslak Julie H. Post Martin G. Tolsgaard Sten Rasmussen Mathias M. Purup Mikkel L. Friis Simulation-based assessment of upper abdominal ultrasound skills BMC Medical Education Simulation-based ultrasound training Validity evidence Ultrasound assessment |
title | Simulation-based assessment of upper abdominal ultrasound skills |
title_full | Simulation-based assessment of upper abdominal ultrasound skills |
title_fullStr | Simulation-based assessment of upper abdominal ultrasound skills |
title_full_unstemmed | Simulation-based assessment of upper abdominal ultrasound skills |
title_short | Simulation-based assessment of upper abdominal ultrasound skills |
title_sort | simulation based assessment of upper abdominal ultrasound skills |
topic | Simulation-based ultrasound training Validity evidence Ultrasound assessment |
url | https://doi.org/10.1186/s12909-023-05018-1 |
work_keys_str_mv | AT kristinaeteslak simulationbasedassessmentofupperabdominalultrasoundskills AT juliehpost simulationbasedassessmentofupperabdominalultrasoundskills AT martingtolsgaard simulationbasedassessmentofupperabdominalultrasoundskills AT stenrasmussen simulationbasedassessmentofupperabdominalultrasoundskills AT mathiasmpurup simulationbasedassessmentofupperabdominalultrasoundskills AT mikkellfriis simulationbasedassessmentofupperabdominalultrasoundskills |