Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified
The rise in popularity and use of cognitive diagnostic models (CDMs) in educational research are partly motivated by the models’ ability to provide diagnostic information regarding students’ strengths and weaknesses in a variety of content areas. An important step to ensure appropriate interpretatio...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-05-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/article/10.3389/fpsyg.2018.00696/full |
| _version_ | 1818352508059254784 |
|---|---|
| author | Dubravka Svetina Yanan Feng Justin Paulsen Montserrat Valdivia Arturo Valdivia Shenghai Dai |
| author_facet | Dubravka Svetina Yanan Feng Justin Paulsen Montserrat Valdivia Arturo Valdivia Shenghai Dai |
| author_sort | Dubravka Svetina |
| collection | DOAJ |
| description | The rise in popularity and use of cognitive diagnostic models (CDMs) in educational research are partly motivated by the models’ ability to provide diagnostic information regarding students’ strengths and weaknesses in a variety of content areas. An important step to ensure appropriate interpretations from CDMs is to investigate differential item functioning (DIF). To this end, the current simulation study examined the performance of three methods to detect DIF in CDMs, with particular emphasis on the impact of Q-matrix misspecification on methods’ performance. Results illustrated that logistic regression and Mantel–Haenszel had better control of Type I error than the Wald test; however, high power rates were found using logistic regression and Wald methods, only. In addition to the tradeoff between Type I error control and acceptable power, our results suggested that Q-matrix complexity and item structures yield different results for different methods, presenting a more complex picture of the methods’ performance. Finally, implications and future directions are discussed. |
| first_indexed | 2024-12-13T18:54:45Z |
| format | Article |
| id | doaj.art-3296b7e49bd74513a885231a4fe7df5f |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-13T18:54:45Z |
| publishDate | 2018-05-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-3296b7e49bd74513a885231a4fe7df5f2022-12-21T23:34:50ZengFrontiers Media S.A.Frontiers in Psychology1664-10782018-05-01910.3389/fpsyg.2018.00696355597Examining DIF in the Context of CDMs When the Q-Matrix Is MisspecifiedDubravka Svetina0Yanan Feng1Justin Paulsen2Montserrat Valdivia3Arturo Valdivia4Shenghai Dai5Department of Counseling and Educational Psychology, Indiana University Bloomington, Bloomington, IN, United StatesDepartment of Counseling and Educational Psychology, Indiana University Bloomington, Bloomington, IN, United StatesDepartment of Counseling and Educational Psychology, Indiana University Bloomington, Bloomington, IN, United StatesDepartment of Counseling and Educational Psychology, Indiana University Bloomington, Bloomington, IN, United StatesDepartment of Statistics, College of Arts and Sciences, Indiana University Bloomington, Bloomington, IN, United StatesEducational Psychology, College of Education, Washington State University, Pullman, WA, United StatesThe rise in popularity and use of cognitive diagnostic models (CDMs) in educational research are partly motivated by the models’ ability to provide diagnostic information regarding students’ strengths and weaknesses in a variety of content areas. An important step to ensure appropriate interpretations from CDMs is to investigate differential item functioning (DIF). To this end, the current simulation study examined the performance of three methods to detect DIF in CDMs, with particular emphasis on the impact of Q-matrix misspecification on methods’ performance. Results illustrated that logistic regression and Mantel–Haenszel had better control of Type I error than the Wald test; however, high power rates were found using logistic regression and Wald methods, only. In addition to the tradeoff between Type I error control and acceptable power, our results suggested that Q-matrix complexity and item structures yield different results for different methods, presenting a more complex picture of the methods’ performance. Finally, implications and future directions are discussed.http://journal.frontiersin.org/article/10.3389/fpsyg.2018.00696/fullcognitive diagnostic modelsdifferential item functioningQ-matrix misspecificationtest biasvalidity |
| spellingShingle | Dubravka Svetina Yanan Feng Justin Paulsen Montserrat Valdivia Arturo Valdivia Shenghai Dai Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified Frontiers in Psychology cognitive diagnostic models differential item functioning Q-matrix misspecification test bias validity |
| title | Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified |
| title_full | Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified |
| title_fullStr | Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified |
| title_full_unstemmed | Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified |
| title_short | Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified |
| title_sort | examining dif in the context of cdms when the q matrix is misspecified |
| topic | cognitive diagnostic models differential item functioning Q-matrix misspecification test bias validity |
| url | http://journal.frontiersin.org/article/10.3389/fpsyg.2018.00696/full |
| work_keys_str_mv | AT dubravkasvetina examiningdifinthecontextofcdmswhentheqmatrixismisspecified AT yananfeng examiningdifinthecontextofcdmswhentheqmatrixismisspecified AT justinpaulsen examiningdifinthecontextofcdmswhentheqmatrixismisspecified AT montserratvaldivia examiningdifinthecontextofcdmswhentheqmatrixismisspecified AT arturovaldivia examiningdifinthecontextofcdmswhentheqmatrixismisspecified AT shenghaidai examiningdifinthecontextofcdmswhentheqmatrixismisspecified |