Contrasting multistage and computer-based testing: score accuracy and aberrant responding
The goal of the present study was to compare and contrast the efficacy of a multistage testing (MST) design using three paths compared to a traditional computer-based testing (CBT) approach involving items across all ability levels. Participants were n = 627 individuals who were subjected to both a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-12-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1288177/full |
| _version_ | 1797404548863623168 |
|---|---|
| author | Georgios Sideridis Georgios Sideridis Hanan Ghamdi Omar Zamil |
| author_facet | Georgios Sideridis Georgios Sideridis Hanan Ghamdi Omar Zamil |
| author_sort | Georgios Sideridis |
| collection | DOAJ |
| description | The goal of the present study was to compare and contrast the efficacy of a multistage testing (MST) design using three paths compared to a traditional computer-based testing (CBT) approach involving items across all ability levels. Participants were n = 627 individuals who were subjected to both a computer-based testing (CBT) instrument and a measure constructed using multistage testing to route individuals of low, middle, and high ability to content that was respective to their ability level. Comparisons between the medium of testing involved person ability accuracy estimates and evaluation of aberrant responding. The results indicated that MST assessments deviated markedly from CBT assessments, especially for low- and high-ability individuals. Test score accuracy was higher overall in MST compared to CBT, although error of measurement was enhanced for high-ability individuals during MST compared to CBT. Evaluating response patterns indicated significant amounts of Guttman-related errors during CBT compared to MST using person-fit aberrant response indicators. It was concluded that MST is associated with significant benefits compared to CBT. |
| first_indexed | 2024-03-09T02:56:36Z |
| format | Article |
| id | doaj.art-f4354d5ac7744e8a804b2a4081fc6e13 |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-03-09T02:56:36Z |
| publishDate | 2023-12-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-f4354d5ac7744e8a804b2a4081fc6e132023-12-05T04:56:04ZengFrontiers Media S.A.Frontiers in Psychology1664-10782023-12-011410.3389/fpsyg.2023.12881771288177Contrasting multistage and computer-based testing: score accuracy and aberrant respondingGeorgios Sideridis0Georgios Sideridis1Hanan Ghamdi2Omar Zamil3Boston Children’s Hospital, Harvard Medical School, Boston, MA, United StatesDepartment of Primary Education, National and Kapodistrian University of Athens, Athens, GreeceEducation and Training Evaluation Commission, Riyadh, Saudi ArabiaEducation and Training Evaluation Commission, Riyadh, Saudi ArabiaThe goal of the present study was to compare and contrast the efficacy of a multistage testing (MST) design using three paths compared to a traditional computer-based testing (CBT) approach involving items across all ability levels. Participants were n = 627 individuals who were subjected to both a computer-based testing (CBT) instrument and a measure constructed using multistage testing to route individuals of low, middle, and high ability to content that was respective to their ability level. Comparisons between the medium of testing involved person ability accuracy estimates and evaluation of aberrant responding. The results indicated that MST assessments deviated markedly from CBT assessments, especially for low- and high-ability individuals. Test score accuracy was higher overall in MST compared to CBT, although error of measurement was enhanced for high-ability individuals during MST compared to CBT. Evaluating response patterns indicated significant amounts of Guttman-related errors during CBT compared to MST using person-fit aberrant response indicators. It was concluded that MST is associated with significant benefits compared to CBT.https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1288177/fullmultistage testingitem response theoryperson fit statisticsaberrant respondingguessingcarelessness |
| spellingShingle | Georgios Sideridis Georgios Sideridis Hanan Ghamdi Omar Zamil Contrasting multistage and computer-based testing: score accuracy and aberrant responding Frontiers in Psychology multistage testing item response theory person fit statistics aberrant responding guessing carelessness |
| title | Contrasting multistage and computer-based testing: score accuracy and aberrant responding |
| title_full | Contrasting multistage and computer-based testing: score accuracy and aberrant responding |
| title_fullStr | Contrasting multistage and computer-based testing: score accuracy and aberrant responding |
| title_full_unstemmed | Contrasting multistage and computer-based testing: score accuracy and aberrant responding |
| title_short | Contrasting multistage and computer-based testing: score accuracy and aberrant responding |
| title_sort | contrasting multistage and computer based testing score accuracy and aberrant responding |
| topic | multistage testing item response theory person fit statistics aberrant responding guessing carelessness |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1288177/full |
| work_keys_str_mv | AT georgiossideridis contrastingmultistageandcomputerbasedtestingscoreaccuracyandaberrantresponding AT georgiossideridis contrastingmultistageandcomputerbasedtestingscoreaccuracyandaberrantresponding AT hananghamdi contrastingmultistageandcomputerbasedtestingscoreaccuracyandaberrantresponding AT omarzamil contrastingmultistageandcomputerbasedtestingscoreaccuracyandaberrantresponding |