Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study
Abstract Background A lumbar puncture procedure’s success depends on a competent physician minimizing the risk of failing to get a sample and avoiding complications such as post-dural headache. A new virtual-reality simulator might be helpful in deciding when a physician is competent to perform lumb...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-10-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12909-023-04806-z |
| _version_ | 1797636811096326144 |
|---|---|
| author | Sujun Xie Søren Grimstrup Leizl Joy Nayahangan Zheng Wang Xing Wan Lars Konge |
| author_facet | Sujun Xie Søren Grimstrup Leizl Joy Nayahangan Zheng Wang Xing Wan Lars Konge |
| author_sort | Sujun Xie |
| collection | DOAJ |
| description | Abstract Background A lumbar puncture procedure’s success depends on a competent physician minimizing the risk of failing to get a sample and avoiding complications such as post-dural headache. A new virtual-reality simulator might be helpful in deciding when a physician is competent to perform lumbar puncture. We aimed to investigate validity evidence for a simulator-based test in lumbar puncture and establish a pass/fail standard to allow a mastery learning training program. Methods Validity evidence was investigated using Messick’s framework by including participants who were novices, intermediates, or experienced in lumbar puncture. Each participant performed two lumbar puncture procedures on the simulator, and fifty-nine predefined simulator metrics were automatically recorded. Cronbach’s alpha was used to explore internal consistency reliability. Intergroup comparisons were made using independent sample t-tests with Tukey’s correction for multiple comparisons. The learning effect was explored using paired sample t-test analysis, and a pass/fail standard was established using the contrasting groups’ method. Results 73 novices, 18 intermediates, and 19 physicians performed the test resulting in a total of 220 procedures. 25 metrics (42.4%) had good discriminatory ability, and the reliability of these metrics was good, Cronbach’s α = 0.81. The experienced physicians were significantly better than the novices (18.3 vs. 13.3, p < 0.001), and the pass/fail standard was established at 16 points. This standard resulted in 22 (30.1%) novices passing (i.e., false positives) and 5 (26.3%) physicians failing (i.e., false negatives). Conclusion This study provides validity evidence for a simulator-based test of lumbar puncture competence. The test can help ensure basic competence at the end of a simulation-based training program for trainees, i.e., a mastery learning training program. |
| first_indexed | 2024-03-11T12:39:31Z |
| format | Article |
| id | doaj.art-44870abfd029440d8bd5c2e7570f5773 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-03-11T12:39:31Z |
| publishDate | 2023-10-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-44870abfd029440d8bd5c2e7570f57732023-11-05T12:21:35ZengBMCBMC Medical Education1472-69202023-10-012311810.1186/s12909-023-04806-zUsing a novel virtual-reality simulator to assess performance in lumbar puncture: a validation studySujun Xie0Søren Grimstrup1Leizl Joy Nayahangan2Zheng Wang3Xing Wan4Lars Konge5Guangzhou University of Chinese MedicineCopenhagen Academy for Medical Education and Simulation (CAMES), Center for Human Resources and EducationCopenhagen Academy for Medical Education and Simulation (CAMES), Center for Human Resources and EducationGuangdong Academy for Medical Simulation (GAMS)Guangzhou University of Chinese MedicineGuangdong Academy for Medical Simulation (GAMS)Abstract Background A lumbar puncture procedure’s success depends on a competent physician minimizing the risk of failing to get a sample and avoiding complications such as post-dural headache. A new virtual-reality simulator might be helpful in deciding when a physician is competent to perform lumbar puncture. We aimed to investigate validity evidence for a simulator-based test in lumbar puncture and establish a pass/fail standard to allow a mastery learning training program. Methods Validity evidence was investigated using Messick’s framework by including participants who were novices, intermediates, or experienced in lumbar puncture. Each participant performed two lumbar puncture procedures on the simulator, and fifty-nine predefined simulator metrics were automatically recorded. Cronbach’s alpha was used to explore internal consistency reliability. Intergroup comparisons were made using independent sample t-tests with Tukey’s correction for multiple comparisons. The learning effect was explored using paired sample t-test analysis, and a pass/fail standard was established using the contrasting groups’ method. Results 73 novices, 18 intermediates, and 19 physicians performed the test resulting in a total of 220 procedures. 25 metrics (42.4%) had good discriminatory ability, and the reliability of these metrics was good, Cronbach’s α = 0.81. The experienced physicians were significantly better than the novices (18.3 vs. 13.3, p < 0.001), and the pass/fail standard was established at 16 points. This standard resulted in 22 (30.1%) novices passing (i.e., false positives) and 5 (26.3%) physicians failing (i.e., false negatives). Conclusion This study provides validity evidence for a simulator-based test of lumbar puncture competence. The test can help ensure basic competence at the end of a simulation-based training program for trainees, i.e., a mastery learning training program.https://doi.org/10.1186/s12909-023-04806-zLumbar punctureVirtual realityAssessmentValidityMastery learning |
| spellingShingle | Sujun Xie Søren Grimstrup Leizl Joy Nayahangan Zheng Wang Xing Wan Lars Konge Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study BMC Medical Education Lumbar puncture Virtual reality Assessment Validity Mastery learning |
| title | Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study |
| title_full | Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study |
| title_fullStr | Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study |
| title_full_unstemmed | Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study |
| title_short | Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study |
| title_sort | using a novel virtual reality simulator to assess performance in lumbar puncture a validation study |
| topic | Lumbar puncture Virtual reality Assessment Validity Mastery learning |
| url | https://doi.org/10.1186/s12909-023-04806-z |
| work_keys_str_mv | AT sujunxie usinganovelvirtualrealitysimulatortoassessperformanceinlumbarpunctureavalidationstudy AT sørengrimstrup usinganovelvirtualrealitysimulatortoassessperformanceinlumbarpunctureavalidationstudy AT leizljoynayahangan usinganovelvirtualrealitysimulatortoassessperformanceinlumbarpunctureavalidationstudy AT zhengwang usinganovelvirtualrealitysimulatortoassessperformanceinlumbarpunctureavalidationstudy AT xingwan usinganovelvirtualrealitysimulatortoassessperformanceinlumbarpunctureavalidationstudy AT larskonge usinganovelvirtualrealitysimulatortoassessperformanceinlumbarpunctureavalidationstudy |