Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece
Computerized adaptive testing (CAT) technology is widely used in a variety of licensing and certification examinations administered to health professionals in the United States. Many more countries worldwide are expected to adopt CAT for their national licensing examinations for health professionals...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Korea Health Personnel Licensing Examination Institute
2018-08-01
|
| Series: | Journal of Educational Evaluation for Health Professions |
| Subjects: | |
| Online Access: | http://www.jeehp.org/upload/jeehp-15-20.pdf |
| _version_ | 1797765125924454400 |
|---|---|
| author | Kyung (Chris) Tyek Han |
| author_facet | Kyung (Chris) Tyek Han |
| author_sort | Kyung (Chris) Tyek Han |
| collection | DOAJ |
| description | Computerized adaptive testing (CAT) technology is widely used in a variety of licensing and certification examinations administered to health professionals in the United States. Many more countries worldwide are expected to adopt CAT for their national licensing examinations for health professionals due to its reduced test time and more accurate estimation of a test-taker’s performance ability. Continuous improvements to CAT algorithms promote the stability and reliability of the results of such examinations. For this reason, conducting simulation studies is a critically important component of evaluating the design of CAT programs and their implementation. This report introduces the principles of SimulCAT, a software program developed for conducting CAT simulation studies. The key evaluation criteria for CAT simulation studies are explained and some guidelines are offered for practitioners and test developers. A step-by-step tutorial example of a SimulCAT run is also presented. The SimulCAT program supports most of the methods used for the 3 key components of item selection in CAT: the item selection criterion, item exposure control, and content balancing. Methods for determining the test length (fixed or variable) and score estimation algorithms are also covered. The simulation studies presented include output files for the response string, item use, standard error of estimation, Newton-Raphson iteration information, theta estimation, the full response matrix, and the true standard error of estimation. In CAT simulations, one condition cannot be generalized to another; therefore, it is recommended that practitioners perform CAT simulation studies in each stage of CAT development. |
| first_indexed | 2024-03-12T20:06:34Z |
| format | Article |
| id | doaj.art-4936d689334e4f938995b902b1d27d52 |
| institution | Directory Open Access Journal |
| issn | 1975-5937 |
| language | English |
| last_indexed | 2024-03-12T20:06:34Z |
| publishDate | 2018-08-01 |
| publisher | Korea Health Personnel Licensing Examination Institute |
| record_format | Article |
| series | Journal of Educational Evaluation for Health Professions |
| spelling | doaj.art-4936d689334e4f938995b902b1d27d522023-08-02T02:04:42ZengKorea Health Personnel Licensing Examination InstituteJournal of Educational Evaluation for Health Professions1975-59372018-08-011510.3352/jeehp.2018.15.20290Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional pieceKyung (Chris) Tyek HanComputerized adaptive testing (CAT) technology is widely used in a variety of licensing and certification examinations administered to health professionals in the United States. Many more countries worldwide are expected to adopt CAT for their national licensing examinations for health professionals due to its reduced test time and more accurate estimation of a test-taker’s performance ability. Continuous improvements to CAT algorithms promote the stability and reliability of the results of such examinations. For this reason, conducting simulation studies is a critically important component of evaluating the design of CAT programs and their implementation. This report introduces the principles of SimulCAT, a software program developed for conducting CAT simulation studies. The key evaluation criteria for CAT simulation studies are explained and some guidelines are offered for practitioners and test developers. A step-by-step tutorial example of a SimulCAT run is also presented. The SimulCAT program supports most of the methods used for the 3 key components of item selection in CAT: the item selection criterion, item exposure control, and content balancing. Methods for determining the test length (fixed or variable) and score estimation algorithms are also covered. The simulation studies presented include output files for the response string, item use, standard error of estimation, Newton-Raphson iteration information, theta estimation, the full response matrix, and the true standard error of estimation. In CAT simulations, one condition cannot be generalized to another; therefore, it is recommended that practitioners perform CAT simulation studies in each stage of CAT development.http://www.jeehp.org/upload/jeehp-15-20.pdfadaptive testingsimulationcomputer software |
| spellingShingle | Kyung (Chris) Tyek Han Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece Journal of Educational Evaluation for Health Professions adaptive testing simulation computer software |
| title | Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece |
| title_full | Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece |
| title_fullStr | Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece |
| title_full_unstemmed | Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece |
| title_short | Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece |
| title_sort | conducting simulation studies for computerized adaptive testing using simulcat an instructional piece |
| topic | adaptive testing simulation computer software |
| url | http://www.jeehp.org/upload/jeehp-15-20.pdf |
| work_keys_str_mv | AT kyungchristyekhan conductingsimulationstudiesforcomputerizedadaptivetestingusingsimulcataninstructionalpiece |