A Method for Evaluating Mode Effects in Mixed-mode Surveys

Abstract

Survey designs in which data from different groups of respondents are collected by different survey modes have become increasingly popular. However, such mixed-mode (MM) designs lead to a confounding of selection effects and measurement effects (measurement error) caused by mode differences. Consequently, MM data have poor quality. Nevertheless, comparing MM data with data from a comparable single-mode survey allows researchers to measure selection effects and measurement effects separately. The authors develop a method to evaluate mode effects and illustrate this method with data from a Dutch MM experiment within the European Social Survey program. In this experiment, respondents could choose between three modes: a Web survey, a telephone interview, or a face-to-face interview. Mode effects on three political variables are evaluated: interest in politics, perceived complexity of politics, and voter turnout in the last national election.