Comparison of the quality estimates in a mixed-mode and a unimode design: an experiment from the European Social Survey

摘要

In the frame of the European Social Survey (ESS), a series of experiments were conducted to investigate if and how the ESS might move from the single face-to-face survey to a mixed-mode design. In order to determine this, many aspects have to be considered. As the ESS wants to maintain the possibility to compare its data across countries and across time, one of the requirements to introduce a mixed-mode design is that it leads to a similar data quality as the current unimode face-to-face design. In this study, we define the quality as the strength of the relationship between the latent concept of interest and the observed answers. Analyzing the experiment done in parallel of the ESS round 6 (2012-2013) in Estonia and the UK, we find that the quality is similar in the unimode and mixed-mode designs, at least for given scales. This is true both for single items and for composite scores. Therefore, standardized relationships in the main ESS round 6 and the mixed-mode experiments can be compared. Besides, for the composite scores, we also find metric and scalar invariance, meaning that unstandardized relationships and means can be compared for the two concepts tested across the unimode and the mixed-mode designs too.