Composite Reliability of Workplace Based Assessment of International Medical Graduates
Introduction All developed countries depend on International Medical Graduates (IMGs) to complement their workforce. However, the assessment of their fitness to practice and acculturation into the new system can be challenging. To improve this, we introduced Workplace Based Assessment (WBA), usin...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
F1000 Research Ltd
2021-04-01
|
| Series: | MedEdPublish |
| Subjects: | |
| Online Access: | https://www.mededpublish.org/Manuscripts/4055 |
| _version_ | 1818348344541446144 |
|---|---|
| author | Balakrishnan Nair Joyce M. W. Moonen – van Loon Mulavana Parvathy Cees P. M. van der Vleuten |
| author_facet | Balakrishnan Nair Joyce M. W. Moonen – van Loon Mulavana Parvathy Cees P. M. van der Vleuten |
| author_sort | Balakrishnan Nair |
| collection | DOAJ |
| description | Introduction
All developed countries depend on International Medical Graduates (IMGs) to complement their workforce. However, the assessment of their fitness to practice and acculturation into the new system can be challenging. To improve this, we introduced Workplace Based Assessment (WBA), using a programmatic philosophy. This paper reports the reliability of this new approach.
Method
Over the past 10 years, we have assessed over 250 IMGs, each cohort assessed over a 6-month period. We used Mini-Cex, Case Based Discussions (CBD) and Multi-Source Feedback (MSF) to assess them. We analysed the reliability of each tool and the composite reliability of 12 Mini-Cex, 5 CBDs and 12 MSF assessments in the tool kit.
Results
A reliability coefficient of 0.78 with a SEM of 0.19 was obtained for the sample of 236 IMGs. We found the MSF to be the most reliable tool. By adding one more MSF to the assessment on two occasions, we can reach a reliability of 0.8 and SEM of 0.18.
Conclusions
The current assessment methodology has acceptable reliability. By increasing the MSF, we can improve the reliability. The lessons from this study are generalisable to IMG assessment and other medical education programs. |
| first_indexed | 2024-12-13T17:48:34Z |
| format | Article |
| id | doaj.art-2da6f8959b35480f8f1f78e9380e0ab8 |
| institution | Directory Open Access Journal |
| issn | 2312-7996 |
| language | English |
| last_indexed | 2024-12-13T17:48:34Z |
| publishDate | 2021-04-01 |
| publisher | F1000 Research Ltd |
| record_format | Article |
| series | MedEdPublish |
| spelling | doaj.art-2da6f8959b35480f8f1f78e9380e0ab82022-12-21T23:36:34ZengF1000 Research LtdMedEdPublish2312-79962021-04-01101Composite Reliability of Workplace Based Assessment of International Medical GraduatesBalakrishnan Nair0Joyce M. W. Moonen – van Loon1Mulavana Parvathy2Cees P. M. van der Vleuten3University of Newcastle, Centre for Medical Professional DevelopmentFaculty of Health, Medicine and Life Sciences, Maastricht UniversityCentre for Medical Professional Development, HNE Health, NewcastleFaculty of Health, Medicine and Life Sciences, Maastricht UniversityIntroduction All developed countries depend on International Medical Graduates (IMGs) to complement their workforce. However, the assessment of their fitness to practice and acculturation into the new system can be challenging. To improve this, we introduced Workplace Based Assessment (WBA), using a programmatic philosophy. This paper reports the reliability of this new approach. Method Over the past 10 years, we have assessed over 250 IMGs, each cohort assessed over a 6-month period. We used Mini-Cex, Case Based Discussions (CBD) and Multi-Source Feedback (MSF) to assess them. We analysed the reliability of each tool and the composite reliability of 12 Mini-Cex, 5 CBDs and 12 MSF assessments in the tool kit. Results A reliability coefficient of 0.78 with a SEM of 0.19 was obtained for the sample of 236 IMGs. We found the MSF to be the most reliable tool. By adding one more MSF to the assessment on two occasions, we can reach a reliability of 0.8 and SEM of 0.18. Conclusions The current assessment methodology has acceptable reliability. By increasing the MSF, we can improve the reliability. The lessons from this study are generalisable to IMG assessment and other medical education programs.https://www.mededpublish.org/Manuscripts/4055Composite reliabilityreliabilityprogrammatic assessmentInternational medical graduatesperformance assessmentWorkplace Based Assessment |
| spellingShingle | Balakrishnan Nair Joyce M. W. Moonen – van Loon Mulavana Parvathy Cees P. M. van der Vleuten Composite Reliability of Workplace Based Assessment of International Medical Graduates MedEdPublish Composite reliability reliability programmatic assessment International medical graduates performance assessment Workplace Based Assessment |
| title | Composite Reliability of Workplace Based Assessment of International Medical Graduates |
| title_full | Composite Reliability of Workplace Based Assessment of International Medical Graduates |
| title_fullStr | Composite Reliability of Workplace Based Assessment of International Medical Graduates |
| title_full_unstemmed | Composite Reliability of Workplace Based Assessment of International Medical Graduates |
| title_short | Composite Reliability of Workplace Based Assessment of International Medical Graduates |
| title_sort | composite reliability of workplace based assessment of international medical graduates |
| topic | Composite reliability reliability programmatic assessment International medical graduates performance assessment Workplace Based Assessment |
| url | https://www.mededpublish.org/Manuscripts/4055 |
| work_keys_str_mv | AT balakrishnannair compositereliabilityofworkplacebasedassessmentofinternationalmedicalgraduates AT joycemwmoonenvanloon compositereliabilityofworkplacebasedassessmentofinternationalmedicalgraduates AT mulavanaparvathy compositereliabilityofworkplacebasedassessmentofinternationalmedicalgraduates AT ceespmvandervleuten compositereliabilityofworkplacebasedassessmentofinternationalmedicalgraduates |