Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion

Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students’ grades depend on these evaluations, change was needed to increase...

Full description

Bibliographic Details
Main Authors: Matthew C. Tews, Robert W. Treat, Maxwell Nanes
Format: Article
Language:English
Published: eScholarship Publishing, University of California 2016-06-01
Series:Western Journal of Emergency Medicine
Subjects:
Online Access:http://escholarship.org/uc/item/77d6j99p
_version_ 1818515915996659712
author Matthew C. Tews
Robert W. Treat
Maxwell Nanes
author_facet Matthew C. Tews
Robert W. Treat
Maxwell Nanes
author_sort Matthew C. Tews
collection DOAJ
description Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students’ grades depend on these evaluations, change was needed to increase form rate of return. We analyzed a new electronic evaluation form and modified completion process to determine if it would increase the completion rate without altering how faculty scored student performance. Methods: During fall 2013, 29 faculty completed paper N=339 evaluations consisting of seven competencies for 33 students. In fall 2014, an electronic evaluation form with the same competencies was designed using an electronic platform and completed N=319 times by 27 faculty using 25 students’ electronic devices. Feedback checkboxes were added to facilitate collection of common comments. Data was analyzed with IBM® SPSS® 21.0 using multi-factor analysis of variance with the students’ global rating (GR) as an outcome. Inter-item reliability was determined with Cronbach alpha. Results: There was a significantly higher completion rate (p=0.001) of 98% electronic vs. 69% paper forms, lower (p=0.001) missed GR rate (1% electronic. vs 12% paper), and higher mean scores (p=0.001) for the GR with the electronic (7.0±1.1) vs. paper (6.8±1.2) form. Feedback checkboxes were completed on every form. The inter-item reliability for electronic and paper forms was each alpha=0.95. Conclusion: The use of a new electronic form and modified completion process for evaluating students at the end of shift demonstrated a higher faculty completion rate, a lower missed data rate, a higher global rating and consistent collection of common feedback. The use of the electronic form and the process for obtaining the information made our end-of-shift evaluation process for students more reliable and provided more accurate, up-to-date information for student feedback and when determining student grades. [West J Emerg Med. 2016;17(4)478-483.]
first_indexed 2024-12-11T00:35:09Z
format Article
id doaj.art-23e2f90d3cff4b359e2e82d876aa0592
institution Directory Open Access Journal
issn 1936-900X
1936-9018
language English
last_indexed 2024-12-11T00:35:09Z
publishDate 2016-06-01
publisher eScholarship Publishing, University of California
record_format Article
series Western Journal of Emergency Medicine
spelling doaj.art-23e2f90d3cff4b359e2e82d876aa05922022-12-22T01:27:10ZengeScholarship Publishing, University of CaliforniaWestern Journal of Emergency Medicine1936-900X1936-90182016-06-0117447848310.5811/westjem.2016.5.29384Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time CompletionMatthew C. Tews0Robert W. Treat1Maxwell Nanes2Medical College of Wisconsin, Department of Emergency Medicine, Milwaukee, Wisconsin Medical College of Wisconsin, Department of Emergency Medicine, Milwaukee, Wisconsin ProHealth Waukesha Memorial Hospital, Emergency Medicine Associates of Waukesha, LLC, Waukesha, Wisconsin Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students’ grades depend on these evaluations, change was needed to increase form rate of return. We analyzed a new electronic evaluation form and modified completion process to determine if it would increase the completion rate without altering how faculty scored student performance. Methods: During fall 2013, 29 faculty completed paper N=339 evaluations consisting of seven competencies for 33 students. In fall 2014, an electronic evaluation form with the same competencies was designed using an electronic platform and completed N=319 times by 27 faculty using 25 students’ electronic devices. Feedback checkboxes were added to facilitate collection of common comments. Data was analyzed with IBM® SPSS® 21.0 using multi-factor analysis of variance with the students’ global rating (GR) as an outcome. Inter-item reliability was determined with Cronbach alpha. Results: There was a significantly higher completion rate (p=0.001) of 98% electronic vs. 69% paper forms, lower (p=0.001) missed GR rate (1% electronic. vs 12% paper), and higher mean scores (p=0.001) for the GR with the electronic (7.0±1.1) vs. paper (6.8±1.2) form. Feedback checkboxes were completed on every form. The inter-item reliability for electronic and paper forms was each alpha=0.95. Conclusion: The use of a new electronic form and modified completion process for evaluating students at the end of shift demonstrated a higher faculty completion rate, a lower missed data rate, a higher global rating and consistent collection of common feedback. The use of the electronic form and the process for obtaining the information made our end-of-shift evaluation process for students more reliable and provided more accurate, up-to-date information for student feedback and when determining student grades. [West J Emerg Med. 2016;17(4)478-483.]http://escholarship.org/uc/item/77d6j99pIncreasing Completion RateM4Emergency MedicineMobile Electronic Platform
spellingShingle Matthew C. Tews
Robert W. Treat
Maxwell Nanes
Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion
Western Journal of Emergency Medicine
Increasing Completion Rate
M4
Emergency Medicine
Mobile Electronic Platform
title Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion
title_full Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion
title_fullStr Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion
title_full_unstemmed Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion
title_short Increasing Completion Rate of an M4 Emergency Medicine Student End-of-Shift Evaluation Using a Mobile Electronic Platform and Real-Time Completion
title_sort increasing completion rate of an m4 emergency medicine student end of shift evaluation using a mobile electronic platform and real time completion
topic Increasing Completion Rate
M4
Emergency Medicine
Mobile Electronic Platform
url http://escholarship.org/uc/item/77d6j99p
work_keys_str_mv AT matthewctews increasingcompletionrateofanm4emergencymedicinestudentendofshiftevaluationusingamobileelectronicplatformandrealtimecompletion
AT robertwtreat increasingcompletionrateofanm4emergencymedicinestudentendofshiftevaluationusingamobileelectronicplatformandrealtimecompletion
AT maxwellnanes increasingcompletionrateofanm4emergencymedicinestudentendofshiftevaluationusingamobileelectronicplatformandrealtimecompletion