Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey
In this article, I consider the problem of "cheating" in political knowledge tests. This problem has been made more pressing by the transition of many surveys to online interviewing, opening up the possibility of looking up the correct answers on the internet. Several methods have been pr...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
European Survey Research Association
2021-04-01
|
| Series: | Survey Research Methods |
| Subjects: | |
| Online Access: | https://ojs.ub.uni-konstanz.de/srm/article/view/7594 |
| _version_ | 1818485991527153664 |
|---|---|
| author | Lionel Marquis |
| author_facet | Lionel Marquis |
| author_sort | Lionel Marquis |
| collection | DOAJ |
| description |
In this article, I consider the problem of "cheating" in political knowledge tests. This problem has been made more pressing by the transition of many surveys to online interviewing, opening up the possibility of looking up the correct answers on the internet. Several methods have been proposed to deal with cheating ex-ante, including self-reports of cheating, control for internet browsing, or time limits. Against this background, “response times” (RTs, i.e., the time taken by respondents to answer a survey question) suggest themselves as a post-hoc, unobtrusive means of detecting cheating. In this paper, I propose a procedure for measuring individual-specific and item-specific RTs, which are then used to identify unusually long but correct answers to knowledge questions as potential cases of cheating. I apply this procedure to the post-electoral survey for the 2015 Swiss national elections. My analysis suggests that extremely slow responses to two out of four questions are definitely suspicious. Accordingly, I propose a method for “correcting” individual knowledge scores and examine its convergent and predictive validity. Based on the finding that a simple revised scale of political knowledge has greater validity than the original additive scale, I conclude that the problem of cheating can be alleviated by using the RT method, which is again summarized in the conclusion to ensure its applicability in empirical research.
|
| first_indexed | 2024-12-10T16:16:58Z |
| format | Article |
| id | doaj.art-3c2e80e692d641f3ac0178436d498434 |
| institution | Directory Open Access Journal |
| issn | 1864-3361 |
| language | English |
| last_indexed | 2024-12-10T16:16:58Z |
| publishDate | 2021-04-01 |
| publisher | European Survey Research Association |
| record_format | Article |
| series | Survey Research Methods |
| spelling | doaj.art-3c2e80e692d641f3ac0178436d4984342022-12-22T01:41:56ZengEuropean Survey Research AssociationSurvey Research Methods1864-33612021-04-0115110.18148/srm/2021.v15i1.7594Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election SurveyLionel Marquis0University of Lausanne In this article, I consider the problem of "cheating" in political knowledge tests. This problem has been made more pressing by the transition of many surveys to online interviewing, opening up the possibility of looking up the correct answers on the internet. Several methods have been proposed to deal with cheating ex-ante, including self-reports of cheating, control for internet browsing, or time limits. Against this background, “response times” (RTs, i.e., the time taken by respondents to answer a survey question) suggest themselves as a post-hoc, unobtrusive means of detecting cheating. In this paper, I propose a procedure for measuring individual-specific and item-specific RTs, which are then used to identify unusually long but correct answers to knowledge questions as potential cases of cheating. I apply this procedure to the post-electoral survey for the 2015 Swiss national elections. My analysis suggests that extremely slow responses to two out of four questions are definitely suspicious. Accordingly, I propose a method for “correcting” individual knowledge scores and examine its convergent and predictive validity. Based on the finding that a simple revised scale of political knowledge has greater validity than the original additive scale, I conclude that the problem of cheating can be alleviated by using the RT method, which is again summarized in the conclusion to ensure its applicability in empirical research. https://ojs.ub.uni-konstanz.de/srm/article/view/7594political knowledgecheatingresponse times |
| spellingShingle | Lionel Marquis Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey Survey Research Methods political knowledge cheating response times |
| title | Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey |
| title_full | Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey |
| title_fullStr | Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey |
| title_full_unstemmed | Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey |
| title_short | Using Response Times to Enhance the Reliability of Political Knowledge Items: An Application to the 2015 Swiss Post-Election Survey |
| title_sort | using response times to enhance the reliability of political knowledge items an application to the 2015 swiss post election survey |
| topic | political knowledge cheating response times |
| url | https://ojs.ub.uni-konstanz.de/srm/article/view/7594 |
| work_keys_str_mv | AT lionelmarquis usingresponsetimestoenhancethereliabilityofpoliticalknowledgeitemsanapplicationtothe2015swisspostelectionsurvey |