Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data
Abstract In cognitive diagnosis assessment (CDA), the impact of misspecified item-attribute relations (or “Q-matrix”) designed by subject-matter experts has been a great challenge to real-world applications. This study examined parameter estimation of the CDA with the expert-designed Q-matrix and tw...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2022-11-01
|
| Series: | Large-scale Assessments in Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40536-022-00138-4 |
| _version_ | 1811215835972239360 |
|---|---|
| author | Jolien Delafontaine Changsheng Chen Jung Yeon Park Wim Van den Noortgate |
| author_facet | Jolien Delafontaine Changsheng Chen Jung Yeon Park Wim Van den Noortgate |
| author_sort | Jolien Delafontaine |
| collection | DOAJ |
| description | Abstract In cognitive diagnosis assessment (CDA), the impact of misspecified item-attribute relations (or “Q-matrix”) designed by subject-matter experts has been a great challenge to real-world applications. This study examined parameter estimation of the CDA with the expert-designed Q-matrix and two refined Q-matrices for international large-scale data. Specifically, the G-DINA model was used to analyze TIMSS data for Grade 8 for five selected countries separately; and the need of a refined Q-matrix specific to the country was investigated. The results suggested that the two refined Q-matrices fitted the data better than the expert-designed Q-matrix, and the stepwise validation method performed better than the nonparametric classification method, resulting in a substantively different classification of students in attribute mastery patterns and different item parameter estimates. The results confirmed that the use of country-specific Q-matrices based on the G-DINA model led to a better fit compared to a universal expert-designed Q-matrix. |
| first_indexed | 2024-04-12T06:29:14Z |
| format | Article |
| id | doaj.art-7160952a45fb471e84f4b251abfa2e5b |
| institution | Directory Open Access Journal |
| issn | 2196-0739 |
| language | English |
| last_indexed | 2024-04-12T06:29:14Z |
| publishDate | 2022-11-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Large-scale Assessments in Education |
| spelling | doaj.art-7160952a45fb471e84f4b251abfa2e5b2022-12-22T03:44:03ZengSpringerOpenLarge-scale Assessments in Education2196-07392022-11-0110113610.1186/s40536-022-00138-4Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale dataJolien Delafontaine0Changsheng Chen1Jung Yeon Park2Wim Van den Noortgate3Faculty of Psychology and Educational Science, KU LeuvenFaculty of Psychology and Educational Science, KU LeuvenFaculty of Psychology and Educational Science, KU LeuvenFaculty of Psychology and Educational Science, KU LeuvenAbstract In cognitive diagnosis assessment (CDA), the impact of misspecified item-attribute relations (or “Q-matrix”) designed by subject-matter experts has been a great challenge to real-world applications. This study examined parameter estimation of the CDA with the expert-designed Q-matrix and two refined Q-matrices for international large-scale data. Specifically, the G-DINA model was used to analyze TIMSS data for Grade 8 for five selected countries separately; and the need of a refined Q-matrix specific to the country was investigated. The results suggested that the two refined Q-matrices fitted the data better than the expert-designed Q-matrix, and the stepwise validation method performed better than the nonparametric classification method, resulting in a substantively different classification of students in attribute mastery patterns and different item parameter estimates. The results confirmed that the use of country-specific Q-matrices based on the G-DINA model led to a better fit compared to a universal expert-designed Q-matrix.https://doi.org/10.1186/s40536-022-00138-4G-DINAQ-matrix refinementStepwise validation methodNonparametric classification methodTIMSS 2011 mathematicsInternational comparison |
| spellingShingle | Jolien Delafontaine Changsheng Chen Jung Yeon Park Wim Van den Noortgate Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data Large-scale Assessments in Education G-DINA Q-matrix refinement Stepwise validation method Nonparametric classification method TIMSS 2011 mathematics International comparison |
| title | Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data |
| title_full | Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data |
| title_fullStr | Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data |
| title_full_unstemmed | Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data |
| title_short | Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data |
| title_sort | using country specific q matrices for cognitive diagnostic assessments with international large scale data |
| topic | G-DINA Q-matrix refinement Stepwise validation method Nonparametric classification method TIMSS 2011 mathematics International comparison |
| url | https://doi.org/10.1186/s40536-022-00138-4 |
| work_keys_str_mv | AT joliendelafontaine usingcountryspecificqmatricesforcognitivediagnosticassessmentswithinternationallargescaledata AT changshengchen usingcountryspecificqmatricesforcognitivediagnosticassessmentswithinternationallargescaledata AT jungyeonpark usingcountryspecificqmatricesforcognitivediagnosticassessmentswithinternationallargescaledata AT wimvandennoortgate usingcountryspecificqmatricesforcognitivediagnosticassessmentswithinternationallargescaledata |