Modeling Response Time and Responses in Multidimensional Health Measurement
This study explored calibrating a large item bank for use in multidimensional health measurement with computerized adaptive testing, using both item responses and response time (RT) information. The Activity Measure for Post-Acute Care is a patient-reported outcomes measure comprised of three correl...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2019-01-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fpsyg.2019.00051/full |
| _version_ | 1818512142926610432 |
|---|---|
| author | Chun Wang David J. Weiss Shiyang Su |
| author_facet | Chun Wang David J. Weiss Shiyang Su |
| author_sort | Chun Wang |
| collection | DOAJ |
| description | This study explored calibrating a large item bank for use in multidimensional health measurement with computerized adaptive testing, using both item responses and response time (RT) information. The Activity Measure for Post-Acute Care is a patient-reported outcomes measure comprised of three correlated scales (Applied Cognition, Daily Activities, and Mobility). All items from each scale are Likert type, so that a respondent chooses a response from an ordered set of four response options. The most appropriate item response theory model for analyzing and scoring these items is the multidimensional graded response model (MGRM). During the field testing of the items, an interviewer read each item to a patient and recorded, on a tablet computer, the patient's responses and the software recorded RTs. Due to the large item bank with over 300 items, data collection was conducted in four batches with a common set of anchor items to link the scale. van der Linden's (2007) hierarchical modeling framework was adopted. Several models, with or without interviewer as a covariate and with or without interaction between interviewer and items, were compared for each batch of data. It was found that the model with the interaction between interviewer and item, when the interaction effect was constrained to be proportional, fit the data best. Therefore, the final hierarchical model with a lognormal model for RT and the MGRM for response data was fitted to all batches of data via a concurrent calibration. Evaluation of parameter estimates revealed that (1) adding response time information did not affect the item parameter estimates and their standard errors significantly; (2) adding response time information helped reduce the standard error of patients' multidimensional latent trait estimates, but adding interviewer as a covariate did not result in further improvement. Implications of the findings for follow up adaptive test delivery design are discussed. |
| first_indexed | 2024-12-10T23:42:50Z |
| format | Article |
| id | doaj.art-d7eb63a3d97946d89656d9e7ae67544f |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-10T23:42:50Z |
| publishDate | 2019-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-d7eb63a3d97946d89656d9e7ae67544f2022-12-22T01:29:00ZengFrontiers Media S.A.Frontiers in Psychology1664-10782019-01-011010.3389/fpsyg.2019.00051420471Modeling Response Time and Responses in Multidimensional Health MeasurementChun Wang0David J. Weiss1Shiyang Su2College of Education, University of Washington, Seattle, WA, United StatesDepartment of Psychology, University of Minnesota, St. Paul, MN, United StatesDepartment of Psychology, University of Central Florida, Orlando, FL, United StatesThis study explored calibrating a large item bank for use in multidimensional health measurement with computerized adaptive testing, using both item responses and response time (RT) information. The Activity Measure for Post-Acute Care is a patient-reported outcomes measure comprised of three correlated scales (Applied Cognition, Daily Activities, and Mobility). All items from each scale are Likert type, so that a respondent chooses a response from an ordered set of four response options. The most appropriate item response theory model for analyzing and scoring these items is the multidimensional graded response model (MGRM). During the field testing of the items, an interviewer read each item to a patient and recorded, on a tablet computer, the patient's responses and the software recorded RTs. Due to the large item bank with over 300 items, data collection was conducted in four batches with a common set of anchor items to link the scale. van der Linden's (2007) hierarchical modeling framework was adopted. Several models, with or without interviewer as a covariate and with or without interaction between interviewer and items, were compared for each batch of data. It was found that the model with the interaction between interviewer and item, when the interaction effect was constrained to be proportional, fit the data best. Therefore, the final hierarchical model with a lognormal model for RT and the MGRM for response data was fitted to all batches of data via a concurrent calibration. Evaluation of parameter estimates revealed that (1) adding response time information did not affect the item parameter estimates and their standard errors significantly; (2) adding response time information helped reduce the standard error of patients' multidimensional latent trait estimates, but adding interviewer as a covariate did not result in further improvement. Implications of the findings for follow up adaptive test delivery design are discussed.https://www.frontiersin.org/article/10.3389/fpsyg.2019.00051/fullresponse timehierarchical modelhealth measurementmultidimensional graded response modelitem response theory (IRT) |
| spellingShingle | Chun Wang David J. Weiss Shiyang Su Modeling Response Time and Responses in Multidimensional Health Measurement Frontiers in Psychology response time hierarchical model health measurement multidimensional graded response model item response theory (IRT) |
| title | Modeling Response Time and Responses in Multidimensional Health Measurement |
| title_full | Modeling Response Time and Responses in Multidimensional Health Measurement |
| title_fullStr | Modeling Response Time and Responses in Multidimensional Health Measurement |
| title_full_unstemmed | Modeling Response Time and Responses in Multidimensional Health Measurement |
| title_short | Modeling Response Time and Responses in Multidimensional Health Measurement |
| title_sort | modeling response time and responses in multidimensional health measurement |
| topic | response time hierarchical model health measurement multidimensional graded response model item response theory (IRT) |
| url | https://www.frontiersin.org/article/10.3389/fpsyg.2019.00051/full |
| work_keys_str_mv | AT chunwang modelingresponsetimeandresponsesinmultidimensionalhealthmeasurement AT davidjweiss modelingresponsetimeandresponsesinmultidimensionalhealthmeasurement AT shiyangsu modelingresponsetimeandresponsesinmultidimensionalhealthmeasurement |