Simulation-based assessment of upper abdominal ultrasound skills

Abstract Background Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasoun...

Full description

Bibliographic Details
Main Authors: Kristina E. Teslak, Julie H. Post, Martin G. Tolsgaard, Sten Rasmussen, Mathias M. Purup, Mikkel L. Friis
Format: Article
Language:English
Published: BMC 2024-01-01
Series:BMC Medical Education
Subjects:
Online Access:https://doi.org/10.1186/s12909-023-05018-1
_version_ 1797363180775669760
author Kristina E. Teslak
Julie H. Post
Martin G. Tolsgaard
Sten Rasmussen
Mathias M. Purup
Mikkel L. Friis
author_facet Kristina E. Teslak
Julie H. Post
Martin G. Tolsgaard
Sten Rasmussen
Mathias M. Purup
Mikkel L. Friis
author_sort Kristina E. Teslak
collection DOAJ
description Abstract Background Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasound skills using an ultrasound simulator. Methods Twenty five novices and five experts were recruited, all of whom completed an assessment program for the evaluation of upper abdominal ultrasound skills on a virtual reality simulator. The program included five modules that assessed different organ systems using automated simulator metrics. We used Messick’s framework to explore the validity evidence of these simulator metrics to determine the contents of a final simulator test. We used the contrasting groups method to establish a pass/fail level for the final simulator test. Results Thirty seven out of 60 metrics were able to discriminate between novices and experts (p < 0.05). The median simulator score of the final simulator test including the metrics with validity evidence was 26.68% (range: 8.1–40.5%) for novices and 85.1% (range: 56.8–91.9%) for experts. The internal structure was assessed by Cronbach alpha (0.93) and intraclass correlation coefficient (0.89). The pass/fail level was determined to be 50.9%. This pass/fail criterion found no passing novices or failing experts. Conclusions This study collected validity evidence for simulation-based assessment of upper abdominal ultrasound examinations, which is the first step toward competency-based training. Future studies may examine how competency-based training in the simulated setting translates into improvements in clinical performances.
first_indexed 2024-03-08T16:17:22Z
format Article
id doaj.art-7070e5c0c36f4bf093fd3f7e505b35da
institution Directory Open Access Journal
issn 1472-6920
language English
last_indexed 2024-03-08T16:17:22Z
publishDate 2024-01-01
publisher BMC
record_format Article
series BMC Medical Education
spelling doaj.art-7070e5c0c36f4bf093fd3f7e505b35da2024-01-07T12:31:10ZengBMCBMC Medical Education1472-69202024-01-012411710.1186/s12909-023-05018-1Simulation-based assessment of upper abdominal ultrasound skillsKristina E. Teslak0Julie H. Post1Martin G. Tolsgaard2Sten Rasmussen3Mathias M. Purup4Mikkel L. Friis5NordSim, Center for Skills Training and Simulation, Aalborg University HospitalNordSim, Center for Skills Training and Simulation, Aalborg University HospitalCopenhagen Academy for Medical Education and Simulation, RigshospitaletDepartment of Clinical Medicine, Aalborg UniversityDepartment of Radiology, Aalborg University HospitalNordSim, Center for Skills Training and Simulation, Aalborg University HospitalAbstract Background Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasound skills using an ultrasound simulator. Methods Twenty five novices and five experts were recruited, all of whom completed an assessment program for the evaluation of upper abdominal ultrasound skills on a virtual reality simulator. The program included five modules that assessed different organ systems using automated simulator metrics. We used Messick’s framework to explore the validity evidence of these simulator metrics to determine the contents of a final simulator test. We used the contrasting groups method to establish a pass/fail level for the final simulator test. Results Thirty seven out of 60 metrics were able to discriminate between novices and experts (p < 0.05). The median simulator score of the final simulator test including the metrics with validity evidence was 26.68% (range: 8.1–40.5%) for novices and 85.1% (range: 56.8–91.9%) for experts. The internal structure was assessed by Cronbach alpha (0.93) and intraclass correlation coefficient (0.89). The pass/fail level was determined to be 50.9%. This pass/fail criterion found no passing novices or failing experts. Conclusions This study collected validity evidence for simulation-based assessment of upper abdominal ultrasound examinations, which is the first step toward competency-based training. Future studies may examine how competency-based training in the simulated setting translates into improvements in clinical performances.https://doi.org/10.1186/s12909-023-05018-1Simulation-based ultrasound trainingValidity evidenceUltrasound assessment
spellingShingle Kristina E. Teslak
Julie H. Post
Martin G. Tolsgaard
Sten Rasmussen
Mathias M. Purup
Mikkel L. Friis
Simulation-based assessment of upper abdominal ultrasound skills
BMC Medical Education
Simulation-based ultrasound training
Validity evidence
Ultrasound assessment
title Simulation-based assessment of upper abdominal ultrasound skills
title_full Simulation-based assessment of upper abdominal ultrasound skills
title_fullStr Simulation-based assessment of upper abdominal ultrasound skills
title_full_unstemmed Simulation-based assessment of upper abdominal ultrasound skills
title_short Simulation-based assessment of upper abdominal ultrasound skills
title_sort simulation based assessment of upper abdominal ultrasound skills
topic Simulation-based ultrasound training
Validity evidence
Ultrasound assessment
url https://doi.org/10.1186/s12909-023-05018-1
work_keys_str_mv AT kristinaeteslak simulationbasedassessmentofupperabdominalultrasoundskills
AT juliehpost simulationbasedassessmentofupperabdominalultrasoundskills
AT martingtolsgaard simulationbasedassessmentofupperabdominalultrasoundskills
AT stenrasmussen simulationbasedassessmentofupperabdominalultrasoundskills
AT mathiasmpurup simulationbasedassessmentofupperabdominalultrasoundskills
AT mikkellfriis simulationbasedassessmentofupperabdominalultrasoundskills