Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment
Mixed mode surveys are presented as a solution to increasing survey costs and decreasing response rates. The disadvantage of such designs is the lack of control over mode effects and the interaction between selection and measurement effects. In a mixed mode survey, measurement effects can put into d...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
GESIS - Leibniz-Institute for the Social Sciences, Mannheim
2016-11-01
|
| Series: | Methoden, Daten, Analysen |
| Subjects: | |
| Online Access: | https://mda.gesis.org/index.php/mda/article/view/2016.011/150 |
| _version_ | 1819228876847448064 |
|---|---|
| author | Caroline Vandenplas Geert Loosveldt Jorre T. A. Vannieuwenhuyze |
| author_facet | Caroline Vandenplas Geert Loosveldt Jorre T. A. Vannieuwenhuyze |
| author_sort | Caroline Vandenplas |
| collection | DOAJ |
| description | Mixed mode surveys are presented as a solution to increasing survey costs and decreasing response rates. The disadvantage of such designs is the lack of control over mode effects and the interaction between selection and measurement effects. In a mixed mode survey, measurement effects can put into doubt data comparability between subgroups, or similarly between waves or rounds of a survey conducted using different modes. To understand the extent of measurement effects, selection and measurement effects between modes have to be disentangled. Almost all techniques to separate these effects depend on covariates that are assumed to be mode-insensitive and to fully explain selection effects. Most of the time, these covariates are sociodemographic variables that might be mode-insensitive, but fail to sufficiently explain selection effects. The aim of this research is to assess the performance of mode preference variables as covariates to evaluate selection and measurement effects between modes.
In 2012, a mixed mode survey – a web questionnaire followed by face-to-face interviews– was conducted alongside the face-to-face European Social Survey in Estonia (Ainsaar et al., 2013). The questionnaire included mode preference items. In this paper, the effects of the trade-offs between the two assumptions on the precision of estimated selection and measurement effects are compared. The results show that while adding the mode preference to the propensity score model seems to increase the explanatory power of web participation, it decreases the correlation between propensity scores and target variables. In addition, the estimated selection and measurement effects do not always fit the expectation that more selection effects are explained and more measurement effects are detected. |
| first_indexed | 2024-12-23T11:04:15Z |
| format | Article |
| id | doaj.art-c5d3b5193f344eca93c9366c6f8f6370 |
| institution | Directory Open Access Journal |
| issn | 1864-6956 2190-4936 |
| language | English |
| last_indexed | 2024-12-23T11:04:15Z |
| publishDate | 2016-11-01 |
| publisher | GESIS - Leibniz-Institute for the Social Sciences, Mannheim |
| record_format | Article |
| series | Methoden, Daten, Analysen |
| spelling | doaj.art-c5d3b5193f344eca93c9366c6f8f63702022-12-21T17:49:33ZengGESIS - Leibniz-Institute for the Social Sciences, MannheimMethoden, Daten, Analysen1864-69562190-49362016-11-0110211913410.12758/mda.2016.011Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode ExperimentCaroline Vandenplas0Geert Loosveldt1Jorre T. A. Vannieuwenhuyze2KU LeuvenKU LeuvenUtrecht UniversityMixed mode surveys are presented as a solution to increasing survey costs and decreasing response rates. The disadvantage of such designs is the lack of control over mode effects and the interaction between selection and measurement effects. In a mixed mode survey, measurement effects can put into doubt data comparability between subgroups, or similarly between waves or rounds of a survey conducted using different modes. To understand the extent of measurement effects, selection and measurement effects between modes have to be disentangled. Almost all techniques to separate these effects depend on covariates that are assumed to be mode-insensitive and to fully explain selection effects. Most of the time, these covariates are sociodemographic variables that might be mode-insensitive, but fail to sufficiently explain selection effects. The aim of this research is to assess the performance of mode preference variables as covariates to evaluate selection and measurement effects between modes. In 2012, a mixed mode survey – a web questionnaire followed by face-to-face interviews– was conducted alongside the face-to-face European Social Survey in Estonia (Ainsaar et al., 2013). The questionnaire included mode preference items. In this paper, the effects of the trade-offs between the two assumptions on the precision of estimated selection and measurement effects are compared. The results show that while adding the mode preference to the propensity score model seems to increase the explanatory power of web participation, it decreases the correlation between propensity scores and target variables. In addition, the estimated selection and measurement effects do not always fit the expectation that more selection effects are explained and more measurement effects are detected.https://mda.gesis.org/index.php/mda/article/view/2016.011/150mixed mode surveysselection effectsmeasurement effectsmode preferanceback-door method |
| spellingShingle | Caroline Vandenplas Geert Loosveldt Jorre T. A. Vannieuwenhuyze Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment Methoden, Daten, Analysen mixed mode surveys selection effects measurement effects mode preferance back-door method |
| title | Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment |
| title_full | Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment |
| title_fullStr | Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment |
| title_full_unstemmed | Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment |
| title_short | Assessing the Use of Mode Preference as a Covariate for the Estimation of Measurement Effects between Modes. A Sequential Mixed Mode Experiment |
| title_sort | assessing the use of mode preference as a covariate for the estimation of measurement effects between modes a sequential mixed mode experiment |
| topic | mixed mode surveys selection effects measurement effects mode preferance back-door method |
| url | https://mda.gesis.org/index.php/mda/article/view/2016.011/150 |
| work_keys_str_mv | AT carolinevandenplas assessingtheuseofmodepreferenceasacovariatefortheestimationofmeasurementeffectsbetweenmodesasequentialmixedmodeexperiment AT geertloosveldt assessingtheuseofmodepreferenceasacovariatefortheestimationofmeasurementeffectsbetweenmodesasequentialmixedmodeexperiment AT jorretavannieuwenhuyze assessingtheuseofmodepreferenceasacovariatefortheestimationofmeasurementeffectsbetweenmodesasequentialmixedmodeexperiment |