September 12, 2018

Hermann Dülmer – The factorial survey: an introduction

Hermann Dülmer,
University of Cologne, 


The central idea of factorials surveys consist in transferring the basic principles of a factorial design (multivariate experimental design) into sample survey. The factorial design consists of varying descriptions of fictitious situations or persons (vignettes) that will be judged by respondents under a particular aspect of interest. Vignettes on trust, for instance, may include gender, skin colour, language skills, denomination, and the employment status of fictitious persons as independent variables (experimental stimuli). The task of the respondents may consist in judging how much he or she would trust a respectively described person. If each respondent evaluates sufficient vignettes, then it becomes possible to estimate the impact of each vignette characteristic via respondent specific regression analysis. If many respondents participate in a factorial survey, then the impact of vignette and respondent characteristics on the answer behaviour can be analysed simultaneously via multilevel analyses. Since the stimuli of the factorial survey are deliberately varied by the researcher, since each vignette serves as “control group” for the other vignettes, and since respondents are assigned randomly to the questionnaires (if different sets of vignettes are used), factorial surveys allow a causal explanation of the observed answer behaviour.

A further advantage of factorial surveys are the detailed vignette descriptions, which ensure that all relevant information is present. Judging concrete vignettes is also much closer to real judgment behaviour in daily life than answering comparably general, most time rather abstract questions. These features contribute to comparably high reliability. By including the factorial design into a sample survey, the factorial survey combines the high internal validity of experiments (evidence of causal relationships) with the high external validity of survey research (generalisability of the results to the broader population).

Since factorial surveys are until now most time not included in standard textbooks, this introduction will embed the logic of factorial surveys into the logic of traditional experiments and thereafter show how to generate and how to analyse a factorial survey. For purposes of illustration, the results of a factorial survey on trust will be presented.

%d bloggers like this: