On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs
Abstract Background This paper extends a recent study by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values of cognitive outcomes in large-scale assessments. Methods Following W...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2018-06-01
|
| Series: | Large-scale Assessments in Education |
| Online Access: | http://link.springer.com/article/10.1186/s40536-018-0059-9 |
| _version_ | 1828274267634008064 |
|---|---|
| author | David Kaplan Dan Su |
| author_facet | David Kaplan Dan Su |
| author_sort | David Kaplan |
| collection | DOAJ |
| description | Abstract Background This paper extends a recent study by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values of cognitive outcomes in large-scale assessments. Methods Following Weirich et al. (Nested multiple imputation in large-scale assessments. In: Large-scale assessments in education, 2. http://www.largescaleassessmentsineducation.com/content/2/1/9, 2014) we examine single + multiple imputation and multiple + multiple imputation methods using predictive mean matching imputation under three different context questionnaire matrix sampling designs: a two-form design studied by Adams et al. (On the use of rotated context questionnaires in conjunction with multilevel item response models. In: Large-scale assessments in education. http://www.largescaleassessmentsineducation.com/content/1/1/5, 2013), a three-form design implemented in PISA 2012, and a partially-balanced incomplete design studied by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016). Results Our results show that the choice of design has a larger impact on the reduction of bias than the choice of imputation method. Specifically, the three-form design used in PISA 2012 yields considerably less bias compared to the two-form design and the partially balanced incomplete design. We further show that the partially balanced incomplete block design produces less bias than the two-form design despite having the same amount of missing data. Conclusions We discuss the results in terms of implications for the design of context questionnaires in large-scale assessments. |
| first_indexed | 2024-04-13T06:33:02Z |
| format | Article |
| id | doaj.art-8f84041774294fdaa59fe65bd55f3c82 |
| institution | Directory Open Access Journal |
| issn | 2196-0739 |
| language | English |
| last_indexed | 2024-04-13T06:33:02Z |
| publishDate | 2018-06-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Large-scale Assessments in Education |
| spelling | doaj.art-8f84041774294fdaa59fe65bd55f3c822022-12-22T02:58:03ZengSpringerOpenLarge-scale Assessments in Education2196-07392018-06-016113110.1186/s40536-018-0059-9On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designsDavid Kaplan0Dan Su1Department of Educational Psychology, University of Wisconsin-MadisonDepartment of Educational Psychology, University of Wisconsin-MadisonAbstract Background This paper extends a recent study by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values of cognitive outcomes in large-scale assessments. Methods Following Weirich et al. (Nested multiple imputation in large-scale assessments. In: Large-scale assessments in education, 2. http://www.largescaleassessmentsineducation.com/content/2/1/9, 2014) we examine single + multiple imputation and multiple + multiple imputation methods using predictive mean matching imputation under three different context questionnaire matrix sampling designs: a two-form design studied by Adams et al. (On the use of rotated context questionnaires in conjunction with multilevel item response models. In: Large-scale assessments in education. http://www.largescaleassessmentsineducation.com/content/1/1/5, 2013), a three-form design implemented in PISA 2012, and a partially-balanced incomplete design studied by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016). Results Our results show that the choice of design has a larger impact on the reduction of bias than the choice of imputation method. Specifically, the three-form design used in PISA 2012 yields considerably less bias compared to the two-form design and the partially balanced incomplete design. We further show that the partially balanced incomplete block design produces less bias than the two-form design despite having the same amount of missing data. Conclusions We discuss the results in terms of implications for the design of context questionnaires in large-scale assessments.http://link.springer.com/article/10.1186/s40536-018-0059-9 |
| spellingShingle | David Kaplan Dan Su On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs Large-scale Assessments in Education |
| title | On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs |
| title_full | On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs |
| title_fullStr | On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs |
| title_full_unstemmed | On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs |
| title_short | On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs |
| title_sort | on imputation for planned missing data in context questionnaires using plausible values a comparison of three designs |
| url | http://link.springer.com/article/10.1186/s40536-018-0059-9 |
| work_keys_str_mv | AT davidkaplan onimputationforplannedmissingdataincontextquestionnairesusingplausiblevaluesacomparisonofthreedesigns AT dansu onimputationforplannedmissingdataincontextquestionnairesusingplausiblevaluesacomparisonofthreedesigns |