Wiss. Koordination

Social, political, and economic issues are typically complex and influenced by many factors. In social science research, it is difficult to disentangle the importance of different factors and to test corresponding theories. This includes survey research when standard survey items are used to measure determinants of normative beliefs, attitudes, preferences, behavioral intentions, etc. Often, all factors appear to be important, and responses are prone to social desirability.

In factorial survey experiments (FSEs), participants are asked to evaluate descriptions of a situation that are experimentally varied on multiple factors/attributes, and participants have to make trade-offs between attributes. FSEs offer therefore a way to disentangle the importance of attributes and reduce social desirability bias. FSEs can also be used to test theories and mechanisms, and they have been applied to a wide range of issues in both the Global North and the Global South, including social acceptance of environmental policies, the perceived fairness of the gender pay gap, and normative beliefs about marriage and child education.

This course will introduce FSEs, and you will become familiar with the theoretical underpinnings and empirical applications of these experiments, as well as learn how to design experiments, collect and analyze FSE data. The course will combine lectures with hands-on sessions using R; for most examples, Stata syntax will be provided. You will have the opportunity to discuss your own research projects/ideas.

Organizational structure of the course

Each day, there will be three hours of input from the lecturers, combined with three hours of practical sessions. The maximum for each input is 45 minutes. The inputs will introduce the main topics such as theoretical foundations, experimental design, questionnaire design and implementation, data analysis and interpretation. All topics will be illustrated with examples, and you will be encouraged to ask questions and contribute to the discussion. The hands-on sessions will allow you to apply these topics in groups or, if preferred, individually using R (for most topics, Stata syntax can be provided). During these sessions, the lecturers will be available for questions and discussion. You will also have the opportunity to discuss your individual projects and/or research problems with the lecturers.

Zielgruppe

You will find the course useful if:

you want to learn more about factorial survey experiments, their potential and pitfalls,

you want to learn how to design and analyze factorial survey experiments,

you are planning a specific application of factorial survey experiments,

you are a postgraduate student or researcher, senior researcher, quantitative social scientist working in the public or private sector.

Lernziel

By the end of the course, you will:

have a good understanding of the (theoretical) foundations of factorial survey experiments,

have a good understanding of the conditions under which factorial survey experiments are applied,

know the foundations of experimental designs for factorial survey experiments,

know how to conduct a factorial survey experiment, including survey design,

be able to build a dataset for factorial survey experiments,

be able to analyze data from factorial survey experiments and interpret model results,

be able to critically reflect on the pitfalls of conducting factorial survey experiments,

be able to critically reflect on the advantages and disadvantages of multifactorial survey experiments more generally.

Voraussetzungen

Basic statistical knowledge, including regression analysis.

Experience in data analysis with a common statistical software (ideally R).

Software requirements

R, RStudio; for experimental design generation, please register for a free SAS OnDemand for Academics account:

https://welcome.oda.sas.com/.

Zeitplan

Day 1: 09:30 to 17:00
Getting to Know the Foundations of Factorial Survey Experiments (FSEs) and the Principles of Experimental Design

Introduction and course overview;

General idea of FSEs; fields of application;

Theoretical foundations of FSEs and causal analysis;

Potential for theory and mechanism testing;

Logic of experimental design and vignette presentation.

Literature

Auspurg, K. and Hinz, T. (2015) 'Multifactorial experiments in surveys: conjoint analysis, choice experiments, and factorial surveys', in T. Wolbring and M. Keuschnigg (eds.), Experimente in den Sozialwissenschaften. Soziale Welt Sonderband 22, Baden-Baden, Germany: Nomos, pp. 291-315.

Liebe, U., Moumouni, I.M., Bigler, C., Ingabire, C. and Bieri, S. (2020) 'Using Factorial Survey Experiments to Measure Attitudes, Social Norms, and Fairness Concerns in Developing Countries', Sociological Methods & Research 49, 161-192.

Liebig, S., Sauer, C. and Friedhoff, S. (2015) 'Using factorial surveys to study justice perceptions: five methodological problems of attitudinal justice research', Social Justice Research 28, 415-434.

Sauer, C., Auspurg, K. and Hinz, T. (2020) 'Designing Multi-Factorial Survey Experiments: Effects of Presentation Style (Text or Table), Answering Scales, and Vignette Order', methods, data, analyses, 14(2), 195-214.

Treischl, E. and Wolbring, T. (2022) 'The Past, Present and Future of Factorial Survey Experiments: A Review for the Social Sciences', methods, data, analyses, 16(2), 30.

Day 2: 09:30 to 17:00
Designing FSE Experiments and Collecting FSE Data

Attribute selection in FSE studies and FSE survey design, including question order;

Experimental design including full factorial, fractional factorial, “efficient” designs with and without attribute interactions;

Methodological effects of FSE designs related to complexity, attribute order, etc.;

Implementation of FSE surveys using different software;

Construction of MFSE datasets including long and wide formats and identification of (serial non-participation).

Literature

Auspurg, K. and Hinz, T. (2015) Factorial Survey Experiments. Sage Series: Quantitative Applications in the Social Sciences. Vol 175. Thousand Oaks CA: Sage [sections on experimental design].

Auspurg, K. and Hinz, T. (2015) Factorial Survey Experiments. Sage Series: Quantitative Applications in the Social Sciences. Vol 175. Thousand Oaks CA: Sage [sections on data collection].

Hainmueller, J., Hangartner, D. and Yamamoto, T. (2015), 'Validating vignette and conjoint survey experiments against real world behavior', Proceedings of the National Academy of Sciences of the United States of America, 112(8), pp. 2395-2400.

Schwitter, N. (2025) 'Using Artificial Intelligence to Generate Visual Vignettes in Factorial Survey Experiments', Social Science Computer Review, online first.

Walzenbach, S. (2019) 'Hiding Sensitive Topics by Design? An Experiment on the Reduction of Social Desirability Bias in Factorial Surveys', Survey Research Methods 13(1), pp. 103-121.

Day 3: 09:30 to 17:00
Analyzing and Presenting FSE Data

Descriptive and bivariate analysis of FSE data;

Random intercept models;

Marginal rates of substitution for FSEs;

Subgroup analysis;

Models including interaction effects with respondent characteristics.

Literature

Auspurg, K. and Hinz, T. (2015) Factorial Survey Experiments. Sage Series: Quantitative Applications in the Social Sciences. Vol 175. Thousand Oaks CA: Sage [sections on data analysis].

Glenk K., Liebe, U., Burns, J. and Thomson, S. (2024) 'Perceived Legitimacy of Agricultural Support Schemes: an Investigation Using Factorial Survey Experiments', Q Open 4, qoae016.

Hox J.J., Kreft, I.G.G. and Hermkens, P.L.J. (1991) 'The analysis of factorial surveys', Sociological Methods & Research 19, pp. 493-510.

Schmidt-Catran, A. W. (2022) 'Factorial surveys with multiple ratings per vignette. A seemingly unrelated multilevel regressions framework', methods, data, analyses 16(2), pp. 335-360.

Weijters, B., Davidov, W. and Baumgartner, H. (2023) ''Analyzing factorial survey data with structural equation models', Sociological Methods & Research 52, pp. 2050-2082.