OSATS Tool for Pediatric Rapid Sequence Intubation
Abstract Introduction Being procedurally competent to perform pediatric rapid sequence intubation (pRSI) is an ACGME requirement for emergency medicine residents. There are few assessment tools to assess this technique with adequate validity evidence. An objective structured assessment of technical...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Association of American Medical Colleges
2014-03-01
|
| Series: | MedEdPORTAL |
| Subjects: | |
| Online Access: | http://www.mededportal.org/doi/10.15766/mep_2374-8265.9727 |
| _version_ | 1798020412873900032 |
|---|---|
| author | Joseph House Suzanne Dooley-Hash Stanley Hamstra M Nypaver |
| author_facet | Joseph House Suzanne Dooley-Hash Stanley Hamstra M Nypaver |
| author_sort | Joseph House |
| collection | DOAJ |
| description | Abstract Introduction Being procedurally competent to perform pediatric rapid sequence intubation (pRSI) is an ACGME requirement for emergency medicine residents. There are few assessment tools to assess this technique with adequate validity evidence. An objective structured assessment of technical skills for pRSI (OSATS-pRSI) was developed and validity evidence was obtained in four of five areas of validity: content, relation to other variables, and inter-rater reliability. Methods This assessment tool was designed to be used to evaluate learner's performance during pPRSI using a standard respiratory distress scenario and a pediatric high-fidelity simulator (Laerdal SimBaby). The scenario was a 7-month old infant who presented to the Emergency Department in respiratory distress. A framework of a previously established objective structured assessment of technical skills (OSATS) tool was modified for pRSI by a panel of pediatric and general emergency medicine physicians. Emergency medicine residents (postgraduate year 1-4) were prospectively enrolled in a pRSI simulation scenario and evaluated by two raters using the modified tool. Results Sessions were videotaped and reviewed by the same raters at least 4 months later. Raters were blinded to their initial rating. Interrater agreement was determined by using the Krippendorff generalized concordance method. Overall interrater agreement for live review was 0.75 (95% CI = 0.72 — 0.78) and for video was 0.79 (95% CI = 0.73 — 0.82). Live review was significantly superior to video review in only one of the OSATS domains (preparation) and was equivalent in the other domains. Intrarater agreement between the live and video evaluation was very good, greater than 0.75 for all raters, with a mean of 0.81 (95% CI = 0.76 — 0.85). Two domains of the modified OSATS instrument (preintubation process and intubation technique) were able to discriminate trainees by level of training and neonatal experience. Discussion The modified OSATS assessment tool demonstrates validity evidence in content, inter- and intra-rater reliability, and relation to other variables. There are limited number of assessment tools for pRSI with validity evidence. This tool has demonstrated validity evidence in content, response process, and inter- and intra-rater reliability. This tool was successfully piloted by 45 emergency medicine residents and showed increased performance in two domains with increasing PGY level. |
| first_indexed | 2024-04-11T16:57:05Z |
| format | Article |
| id | doaj.art-c360fbd024d9493888adaac3bdbedf5f |
| institution | Directory Open Access Journal |
| issn | 2374-8265 |
| language | English |
| last_indexed | 2024-04-11T16:57:05Z |
| publishDate | 2014-03-01 |
| publisher | Association of American Medical Colleges |
| record_format | Article |
| series | MedEdPORTAL |
| spelling | doaj.art-c360fbd024d9493888adaac3bdbedf5f2022-12-22T04:13:14ZengAssociation of American Medical CollegesMedEdPORTAL2374-82652014-03-011010.15766/mep_2374-8265.9727OSATS Tool for Pediatric Rapid Sequence IntubationJoseph House0Suzanne Dooley-Hash1Stanley Hamstra2M Nypaver31 University of Michigan Medical School2 University of Michigan School of Medicine3 Ottawa Skills and Simulation Centre4 University of Michigan Health SystemAbstract Introduction Being procedurally competent to perform pediatric rapid sequence intubation (pRSI) is an ACGME requirement for emergency medicine residents. There are few assessment tools to assess this technique with adequate validity evidence. An objective structured assessment of technical skills for pRSI (OSATS-pRSI) was developed and validity evidence was obtained in four of five areas of validity: content, relation to other variables, and inter-rater reliability. Methods This assessment tool was designed to be used to evaluate learner's performance during pPRSI using a standard respiratory distress scenario and a pediatric high-fidelity simulator (Laerdal SimBaby). The scenario was a 7-month old infant who presented to the Emergency Department in respiratory distress. A framework of a previously established objective structured assessment of technical skills (OSATS) tool was modified for pRSI by a panel of pediatric and general emergency medicine physicians. Emergency medicine residents (postgraduate year 1-4) were prospectively enrolled in a pRSI simulation scenario and evaluated by two raters using the modified tool. Results Sessions were videotaped and reviewed by the same raters at least 4 months later. Raters were blinded to their initial rating. Interrater agreement was determined by using the Krippendorff generalized concordance method. Overall interrater agreement for live review was 0.75 (95% CI = 0.72 — 0.78) and for video was 0.79 (95% CI = 0.73 — 0.82). Live review was significantly superior to video review in only one of the OSATS domains (preparation) and was equivalent in the other domains. Intrarater agreement between the live and video evaluation was very good, greater than 0.75 for all raters, with a mean of 0.81 (95% CI = 0.76 — 0.85). Two domains of the modified OSATS instrument (preintubation process and intubation technique) were able to discriminate trainees by level of training and neonatal experience. Discussion The modified OSATS assessment tool demonstrates validity evidence in content, inter- and intra-rater reliability, and relation to other variables. There are limited number of assessment tools for pRSI with validity evidence. This tool has demonstrated validity evidence in content, response process, and inter- and intra-rater reliability. This tool was successfully piloted by 45 emergency medicine residents and showed increased performance in two domains with increasing PGY level.http://www.mededportal.org/doi/10.15766/mep_2374-8265.9727InfantNeonateSpinal PunctureMethodsPediatricsAssessment Process |
| spellingShingle | Joseph House Suzanne Dooley-Hash Stanley Hamstra M Nypaver OSATS Tool for Pediatric Rapid Sequence Intubation MedEdPORTAL Infant Neonate Spinal Puncture Methods Pediatrics Assessment Process |
| title | OSATS Tool for Pediatric Rapid Sequence Intubation |
| title_full | OSATS Tool for Pediatric Rapid Sequence Intubation |
| title_fullStr | OSATS Tool for Pediatric Rapid Sequence Intubation |
| title_full_unstemmed | OSATS Tool for Pediatric Rapid Sequence Intubation |
| title_short | OSATS Tool for Pediatric Rapid Sequence Intubation |
| title_sort | osats tool for pediatric rapid sequence intubation |
| topic | Infant Neonate Spinal Puncture Methods Pediatrics Assessment Process |
| url | http://www.mededportal.org/doi/10.15766/mep_2374-8265.9727 |
| work_keys_str_mv | AT josephhouse osatstoolforpediatricrapidsequenceintubation AT suzannedooleyhash osatstoolforpediatricrapidsequenceintubation AT stanleyhamstra osatstoolforpediatricrapidsequenceintubation AT mnypaver osatstoolforpediatricrapidsequenceintubation |