Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample
Monitoring of cognitive abilities in large-scale survey research is receiving increasing attention. Conventional cognitive testing, however, is often impractical on a population level highlighting the need for alternative means of cognitive assessment. We evaluated whether response times (RTs) to on...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Journal of Intelligence |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-3200/11/1/3 |
| _version_ | 1797440487322288128 |
|---|---|
| author | Doerte U. Junghaenel Stefan Schneider Bart Orriens Haomiao Jin Pey-Jiuan Lee Arie Kapteyn Erik Meijer Elizabeth Zelinski Raymond Hernandez Arthur A. Stone |
| author_facet | Doerte U. Junghaenel Stefan Schneider Bart Orriens Haomiao Jin Pey-Jiuan Lee Arie Kapteyn Erik Meijer Elizabeth Zelinski Raymond Hernandez Arthur A. Stone |
| author_sort | Doerte U. Junghaenel |
| collection | DOAJ |
| description | Monitoring of cognitive abilities in large-scale survey research is receiving increasing attention. Conventional cognitive testing, however, is often impractical on a population level highlighting the need for alternative means of cognitive assessment. We evaluated whether response times (RTs) to online survey items could be useful to infer cognitive abilities. We analyzed >5 million survey item RTs from >6000 individuals administered over 6.5 years in an internet panel together with cognitive tests (numerical reasoning, verbal reasoning, task switching/inhibitory control). We derived measures of mean RT and intraindividual RT variability from a multilevel location-scale model as well as an expanded version that separated intraindividual RT variability into systematic RT adjustments (variation of RTs with item time intensities) and residual intraindividual RT variability (residual error in RTs). RT measures from the location-scale model showed weak associations with cognitive test scores. However, RT measures from the expanded model explained 22–26% of the variance in cognitive scores and had prospective associations with cognitive assessments over lag-periods of at least 6.5 years (mean RTs), 4.5 years (systematic RT adjustments) and 1 year (residual RT variability). Our findings suggest that RTs in online surveys may be useful for gaining information about cognitive abilities in large-scale survey research. |
| first_indexed | 2024-03-09T12:07:54Z |
| format | Article |
| id | doaj.art-a570c51b30eb43d687ac096841c79292 |
| institution | Directory Open Access Journal |
| issn | 2079-3200 |
| language | English |
| last_indexed | 2024-03-09T12:07:54Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Intelligence |
| spelling | doaj.art-a570c51b30eb43d687ac096841c792922023-11-30T22:55:36ZengMDPI AGJournal of Intelligence2079-32002022-12-01111310.3390/jintelligence11010003Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative SampleDoerte U. Junghaenel0Stefan Schneider1Bart Orriens2Haomiao Jin3Pey-Jiuan Lee4Arie Kapteyn5Erik Meijer6Elizabeth Zelinski7Raymond Hernandez8Arthur A. Stone9Department of Psychology, University of Southern California, Los Angeles, CA 90089, USADepartment of Psychology, University of Southern California, Los Angeles, CA 90089, USACenter for Economic and Social Research, University of Southern California, Los Angeles, CA 90089, USASchool of Health Sciences, University of Surrey, Guildford GU2 7YH, UKCenter for Economic and Social Research, University of Southern California, Los Angeles, CA 90089, USACenter for Economic and Social Research, University of Southern California, Los Angeles, CA 90089, USACenter for Economic and Social Research, University of Southern California, Los Angeles, CA 90089, USALeonard Davis School of Gerontology, University of Southern California, Los Angeles, CA 90089, USACenter for Economic and Social Research, University of Southern California, Los Angeles, CA 90089, USADepartment of Psychology, University of Southern California, Los Angeles, CA 90089, USAMonitoring of cognitive abilities in large-scale survey research is receiving increasing attention. Conventional cognitive testing, however, is often impractical on a population level highlighting the need for alternative means of cognitive assessment. We evaluated whether response times (RTs) to online survey items could be useful to infer cognitive abilities. We analyzed >5 million survey item RTs from >6000 individuals administered over 6.5 years in an internet panel together with cognitive tests (numerical reasoning, verbal reasoning, task switching/inhibitory control). We derived measures of mean RT and intraindividual RT variability from a multilevel location-scale model as well as an expanded version that separated intraindividual RT variability into systematic RT adjustments (variation of RTs with item time intensities) and residual intraindividual RT variability (residual error in RTs). RT measures from the location-scale model showed weak associations with cognitive test scores. However, RT measures from the expanded model explained 22–26% of the variance in cognitive scores and had prospective associations with cognitive assessments over lag-periods of at least 6.5 years (mean RTs), 4.5 years (systematic RT adjustments) and 1 year (residual RT variability). Our findings suggest that RTs in online surveys may be useful for gaining information about cognitive abilities in large-scale survey research.https://www.mdpi.com/2079-3200/11/1/3cognitive abilitiesresponse timessurvey researchself-reportonline surveysresponse time variability |
| spellingShingle | Doerte U. Junghaenel Stefan Schneider Bart Orriens Haomiao Jin Pey-Jiuan Lee Arie Kapteyn Erik Meijer Elizabeth Zelinski Raymond Hernandez Arthur A. Stone Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample Journal of Intelligence cognitive abilities response times survey research self-report online surveys response time variability |
| title | Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample |
| title_full | Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample |
| title_fullStr | Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample |
| title_full_unstemmed | Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample |
| title_short | Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample |
| title_sort | inferring cognitive abilities from response times to web administered survey items in a population representative sample |
| topic | cognitive abilities response times survey research self-report online surveys response time variability |
| url | https://www.mdpi.com/2079-3200/11/1/3 |
| work_keys_str_mv | AT doerteujunghaenel inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT stefanschneider inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT bartorriens inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT haomiaojin inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT peyjiuanlee inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT ariekapteyn inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT erikmeijer inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT elizabethzelinski inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT raymondhernandez inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample AT arthurastone inferringcognitiveabilitiesfromresponsetimestowebadministeredsurveyitemsinapopulationrepresentativesample |