Detecting Differential Item Functioning in 2PL Multistage Assessments
The detection of differential item functioning is crucial for the psychometric evaluation of multistage tests. This paper discusses five approaches presented in the literature: logistic regression, SIBTEST, analytical score-based tests, bootstrap score-based tests, and permutation score-based tests....
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Psych |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2624-8611/5/2/31 |
| _version_ | 1797592854079471616 |
|---|---|
| author | Rudolf Debelak Sebastian Appelbaum Dries Debeer Martin J. Tomasik |
| author_facet | Rudolf Debelak Sebastian Appelbaum Dries Debeer Martin J. Tomasik |
| author_sort | Rudolf Debelak |
| collection | DOAJ |
| description | The detection of differential item functioning is crucial for the psychometric evaluation of multistage tests. This paper discusses five approaches presented in the literature: logistic regression, SIBTEST, analytical score-based tests, bootstrap score-based tests, and permutation score-based tests. First, using an simulation study inspired by a real-life large-scale educational assessment, we compare the five approaches with respect to their type I error rate and their statistical power. Then, we present an application to an empirical data set. We find that all approaches show type I error rates close to the nominal alpha level. Furthermore, all approaches are shown to be sensitive to uniform and non-uniform DIF effects, with the score-based tests showing the highest power. |
| first_indexed | 2024-03-11T02:00:06Z |
| format | Article |
| id | doaj.art-8a4f0d5f0301408887fbd6fb5dd69da6 |
| institution | Directory Open Access Journal |
| issn | 2624-8611 |
| language | English |
| last_indexed | 2024-03-11T02:00:06Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Psych |
| spelling | doaj.art-8a4f0d5f0301408887fbd6fb5dd69da62023-11-18T12:21:10ZengMDPI AGPsych2624-86112023-05-015246147710.3390/psych5020031Detecting Differential Item Functioning in 2PL Multistage AssessmentsRudolf Debelak0Sebastian Appelbaum1Dries Debeer2Martin J. Tomasik3Department for Psychology, University of Zurich, 8050 Zurich, SwitzerlandSchool of Psychology and Psychotherapy, Witten/Herdecke University, 58448 Witten, GermanyFaculty of Psychology and Educational Sciences, Ghent University, 9000 Gent, BelgiumDepartment for Psychology, University of Zurich, 8050 Zurich, SwitzerlandThe detection of differential item functioning is crucial for the psychometric evaluation of multistage tests. This paper discusses five approaches presented in the literature: logistic regression, SIBTEST, analytical score-based tests, bootstrap score-based tests, and permutation score-based tests. First, using an simulation study inspired by a real-life large-scale educational assessment, we compare the five approaches with respect to their type I error rate and their statistical power. Then, we present an application to an empirical data set. We find that all approaches show type I error rates close to the nominal alpha level. Furthermore, all approaches are shown to be sensitive to uniform and non-uniform DIF effects, with the score-based tests showing the highest power.https://www.mdpi.com/2624-8611/5/2/31item response theorydifferential item functioningmultistage testing |
| spellingShingle | Rudolf Debelak Sebastian Appelbaum Dries Debeer Martin J. Tomasik Detecting Differential Item Functioning in 2PL Multistage Assessments Psych item response theory differential item functioning multistage testing |
| title | Detecting Differential Item Functioning in 2PL Multistage Assessments |
| title_full | Detecting Differential Item Functioning in 2PL Multistage Assessments |
| title_fullStr | Detecting Differential Item Functioning in 2PL Multistage Assessments |
| title_full_unstemmed | Detecting Differential Item Functioning in 2PL Multistage Assessments |
| title_short | Detecting Differential Item Functioning in 2PL Multistage Assessments |
| title_sort | detecting differential item functioning in 2pl multistage assessments |
| topic | item response theory differential item functioning multistage testing |
| url | https://www.mdpi.com/2624-8611/5/2/31 |
| work_keys_str_mv | AT rudolfdebelak detectingdifferentialitemfunctioningin2plmultistageassessments AT sebastianappelbaum detectingdifferentialitemfunctioningin2plmultistageassessments AT driesdebeer detectingdifferentialitemfunctioningin2plmultistageassessments AT martinjtomasik detectingdifferentialitemfunctioningin2plmultistageassessments |