Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination
Purpose Computerized adaptive testing (CAT) has been adopted in licensing examinations because it improves the efficiency and accuracy of the tests, as shown in many studies. This simulation study investigated CAT scoring and item selection methods for the Korean Medical Licensing Examination (KMLE)...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Korea Health Personnel Licensing Examination Institute
2018-05-01
|
| Series: | Journal of Educational Evaluation for Health Professions |
| Subjects: | |
| Online Access: | http://www.jeehp.org/upload/jeehp-15-14.pdf |
| _version_ | 1797766014928158720 |
|---|---|
| author | Dong Gi Seo Jeongwook Choi |
| author_facet | Dong Gi Seo Jeongwook Choi |
| author_sort | Dong Gi Seo |
| collection | DOAJ |
| description | Purpose Computerized adaptive testing (CAT) has been adopted in licensing examinations because it improves the efficiency and accuracy of the tests, as shown in many studies. This simulation study investigated CAT scoring and item selection methods for the Korean Medical Licensing Examination (KMLE). Methods This study used a post-hoc (real data) simulation design. The item bank used in this study included all items from the January 2017 KMLE. All CAT algorithms for this study were implemented using the ‘catR’ package in the R program. Results In terms of accuracy, the Rasch and 2-parametric logistic (PL) models performed better than the 3PL model. The ‘modal a posteriori’ and ‘expected a posterior’ methods provided more accurate estimates than maximum likelihood estimation or weighted likelihood estimation. Furthermore, maximum posterior weighted information and minimum expected posterior variance performed better than other item selection methods. In terms of efficiency, the Rasch model is recommended to reduce test length. Conclusion Before implementing live CAT, a simulation study should be performed under varied test conditions. Based on a simulation study, and based on the results, specific scoring and item selection methods should be predetermined. |
| first_indexed | 2024-03-12T20:19:11Z |
| format | Article |
| id | doaj.art-2673cb021ea943f0896f6196bda8dcc2 |
| institution | Directory Open Access Journal |
| issn | 1975-5937 |
| language | English |
| last_indexed | 2024-03-12T20:19:11Z |
| publishDate | 2018-05-01 |
| publisher | Korea Health Personnel Licensing Examination Institute |
| record_format | Article |
| series | Journal of Educational Evaluation for Health Professions |
| spelling | doaj.art-2673cb021ea943f0896f6196bda8dcc22023-08-02T01:03:08ZengKorea Health Personnel Licensing Examination InstituteJournal of Educational Evaluation for Health Professions1975-59372018-05-011510.3352/jeehp.2018.15.14284Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing ExaminationDong Gi SeoJeongwook ChoiPurpose Computerized adaptive testing (CAT) has been adopted in licensing examinations because it improves the efficiency and accuracy of the tests, as shown in many studies. This simulation study investigated CAT scoring and item selection methods for the Korean Medical Licensing Examination (KMLE). Methods This study used a post-hoc (real data) simulation design. The item bank used in this study included all items from the January 2017 KMLE. All CAT algorithms for this study were implemented using the ‘catR’ package in the R program. Results In terms of accuracy, the Rasch and 2-parametric logistic (PL) models performed better than the 3PL model. The ‘modal a posteriori’ and ‘expected a posterior’ methods provided more accurate estimates than maximum likelihood estimation or weighted likelihood estimation. Furthermore, maximum posterior weighted information and minimum expected posterior variance performed better than other item selection methods. In terms of efficiency, the Rasch model is recommended to reduce test length. Conclusion Before implementing live CAT, a simulation study should be performed under varied test conditions. Based on a simulation study, and based on the results, specific scoring and item selection methods should be predetermined.http://www.jeehp.org/upload/jeehp-15-14.pdfalgorithmscomputerskorealogistic modelsresearch design |
| spellingShingle | Dong Gi Seo Jeongwook Choi Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination Journal of Educational Evaluation for Health Professions algorithms computers korea logistic models research design |
| title | Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination |
| title_full | Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination |
| title_fullStr | Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination |
| title_full_unstemmed | Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination |
| title_short | Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination |
| title_sort | post hoc simulation study of computerized adaptive testing for the korean medical licensing examination |
| topic | algorithms computers korea logistic models research design |
| url | http://www.jeehp.org/upload/jeehp-15-14.pdf |
| work_keys_str_mv | AT donggiseo posthocsimulationstudyofcomputerizedadaptivetestingforthekoreanmedicallicensingexamination AT jeongwookchoi posthocsimulationstudyofcomputerizedadaptivetestingforthekoreanmedicallicensingexamination |