Wiss. Koordination

Alisa Remizova

Administrative Koordination

Noemi Hartung

Field Experiments

About

Date:
11.03 - 12.03.2027

Location:
Online via Zoom

General Topics:

Data Analysis, Survey Methodology

Course Level:

Advanced

Format:

Workshop

Software used:

R, Stata

Duration:

2 days

Language:

English

Fees:

Students: 220 €

Academics: 330 €

Commercial: 660 €

Keywords

field experiment, natural field experiment, lost letter/ wallet studies, audit studies, digital field experiments

Additional links

Terms and Conditions
FAQs

Lecturer(s): Johanna Gereke, Leonard Wendering (TA)

About the lecturer - Johanna Gereke

About the lecturer - Leonard Wendering (TA)

Course description

Registration deadline: 22.02.2027

Registration after this date might be possible, but is subject to sufficient demand and availability of places.

Field experiments are essential tools for testing causal relationships in real-world contexts. They are particularly valuable for advancing theory, testing causal claims, and informing social policy, especially with regard to research on discrimination, intergroup relations, and social norms.

This course aims to enhance your research skills in the experimental social sciences by providing the conceptual and analytical tools needed to design and implement field experiments that address specific research questions. By the end of the course, you will be familiar with various types of field experiments, including natural field experiments with confederates, audit and correspondence studies, as well as digital field experiments. The course will cover each stage of conducting a field experiment, from design and randomization to data analysis, while emphasizing important topics such as subgroup analysis, non-compliance, and research ethics.

Organizational structure of the course

The course will consist of two blocks of three hours of instruction per day: one in the morning and one in the afternoon. The lectures will cover essential methods and general considerations for different types of field experiments while discussing a range of empirical examples. The course will also include some hands-on tutorials. You are strongly encouraged to engage actively in these sessions by reading the assigned empirical studies, asking questions, and contributing to discussions. Moreover, you are welcome to bring your own research ideas to work on during the course.

Target group

You will find the course useful if:

you are interested in applying experimental methods to address causal questions in social science research.
you are an advanced master student, PhD student, or postdoctoral researcher in quantitative social sciences.
you aim to develop practical skills for analyzing experimental data, including handling challenges like non-compliance and attrition.
you are concerned about the ethical implications of experimental research and want to learn best practices in research ethics.

Learning objectives

By the end of the course, you will:

have gained an overview of the most common field experimental methods.
know the strengths and weaknesses of the different field experimental methods.
have gained a good understanding of how to design and implement field experiments as well as how to analyze the data and report on the results.
understand the practical and ethical limitations and good research practices (e.g. pre-registration).

Prerequisites

Solid basic statistical knowledge.
Practical experience in data analysis with a common statistical software (Stata or R).

Software and hardware requirements

As a participant, you should bring your own laptop for use in the course. You will be able to use either R or Stata for the coding exercises. The latest version of R is available for free at https://cran.r-project.org/, and RStudio is available at https://www.rstudio.com/. Stata short-term licenses can be provided by GESIS for the duration of the course if needed - please contact us at least two weeks prior to the course. Throughout the course, you will need to be able to install additional user packages on the fly (this requires an internet connection and appropriate writing rights on the local system).

Schedule

Day	Topic(s)
Day 1, 09:00-12:00	Introduction to field experiments: What is causality and how do experiments help causal identification? Introduction and course overview General idea of experimental methods; different types of field experiments Field experiments with confederates Field experiments with passive interventions Lost wallet / lost letter studies Audit Studies and Correspondence Tests Digital Field Experiments with and without platform collaboration Field Experiments with Survey Outcomes Experimental design principles and key concepts Randomization, treatment and control groups Internal vs. external validity, causal inference
	Required reading: Baldassarri, D., & Abascal, M. (2017). Field experiments across the social sciences. Annual review of sociology, 43, 41-73. https://doi.org/10.1146/annurev-soc-073014-112445 Gerber, A. S., & Green, D. P. (2012). Field experiments: Design, analysis, and interpretation. Chapter 1& 2.1 and 2.2 only. Gereke, J., Hellyer, J., & Zschirnt, E. (Forthcoming). Field experiments. In U. Liebe (Ed.), Handbook of Quantitative Methods in Sociology. Edward Elgar Publishing. https://doi.org/10.31219/osf.io/v3kqm Suggested reading: Kalla, J. L., Broockman, D. E., & Sekhon, J. S. (2021). Field experiments with survey outcomes. In J. N. Druckman & D. P. Green, Advances in Experimental Political Science (pp. 56-78). Cambridge University Press. https://doi.org/10.1017/9781108777919.006 Wilke, A. M., & Humphreys, M. Field Experiments, Theory, and External Validity. In L. Curini & R. Franzese (Eds.), SAGE Handbook of Research Methods in Political Science and International Relations. https://doi.org/10.4135/9781526486387.n56 Bertrand, M., & Mullainathan, S. (2004). Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination. American Economic Review, 94(4), 991-1013 https://doi.org/10.1257/0002828042002561 Bjørnshagen, V., & Ugreninov, E. (2021). Disability disadvantage: experimental evidence of hiring discrimination against wheelchair users. European Sociological Review, 37(5), 818-833. http://doi.org/10.1093/esr/jcab004 Chetty, R., Hendren, N., & Katz, L. F. (2016). The effects of exposure to better neighborhoods on children: New evidence from the moving to opportunity experiment. American Economic Review, 106(4), 855-902. https://doi.org/10.1257/aer.20150572 Cohn, A., Maréchal, M. A., Tannenbaum, D., & Zünd, C. L. (2019). Civic honesty around the globe. Science, 365(6448), 70-73. https://doi.org/10.1126/science.aau8712 Gerber, A. S., Green, D. P., & Larimer, C. W. (2008). Social pressure and voter turnout: Evidence from a large-scale field experiment. American political Science review, 102(1), 33-48. https://doi.org/10.1017/S000305540808009X Keizer, K., Lindenberg, S., & Steg, L. (2008). The spreading of disorder. Science, 322(5908), 1681-1685. https://doi.org/10.1126/science.1161405 Liebe, U., Gewinner, J., & Diekmann, A. (2021). Large and persistent effects of green energy defaults in the household and business sectors. Nature Human Behaviour, 5(5), 576-585. https://doi.org/10.1038/s41562-021-01070-3 Mousa, S. (2020). Building social cohesion between Christians and Muslims through soccer in post-ISIS Iraq. Science, 369(6505), 866-870. https://doi.org/10.1126/science.abb3153 Rajkumar, K., Saint-Jacques, G., Bojinov, I., Brynjolfsson, E., & Aral, S. (2022). A causal test of the strength of weak ties. Science, 377(6612), 1304-1310. https://doi.org/10.1126/science.abl4476 Sands, M. L., & de Kadt, D. (2020). Local exposure to inequality raises support of people of low wealth for taxing the wealthy. Nature, 586(7828), 257-261. https://doi.org/10.1038/s41586-020-2763-1 Van de Rijt, A., Kang, S. M., Restivo, M., & Patil, A. (2014). Field experiments of success-breeds-success dynamics. Proceedings of the National Academy of Sciences, 111(19), 6934-6939. https://doi.org/10.1073/pnas.1316836111 Winter, F., & Zhang, N. (2018). Social norm enforcement in ethnically diverse communities. Proceedings of the National Academy of Sciences, 115(11), 2722-2727. https://doi.org/10.1073/pnas.1718309115
Day 1, 13:00-16:00	Randomization strategies and hypothesis testing / Research Ethics Core Assumptions: Randomization, Excludability, Non-interference Sampling and recruitment in field experiments Techniques for sampling in field contexts Hypothesis testing Null vs. alternative hypotheses, Type I and Type II errors Research ethics and challenges in field settings: consent, potential harm, and researchers' responsibilities
	Required reading: Gerber, A. S., & Green, D. P. (2012). Field experiments: Design, analysis, and interpretation. Chapter 2, chapter 3. Coppock, A. 10 Things to Know About Statistical Power. https://egap.org/resource/10-things-to-know-about-statistical-power/ Phillips, T. (2021). Ethics of Field Experiments. Annual Review of Political Science, 24, 277-300. https://doi.org/10.1146/annurev-polisci-041719-101956 Suggested reading: Asiedu, E., Karlan, D., Lambon-Quayefio, M., & Udry, C. (2021). A call for structured ethics appendices in social science papers. Proceedings of the National Academy of Sciences, 118(29), e2024570118. https://doi.org/10.3386/w28393 Auspurg, K., Schneck, A., & Thiel, F. (2020). Different samples, different results? How sampling techniques affect the results of field experiments on ethnic discrimination. Research in Social Stratification and Mobility, 65, 100444. https://doi.org/10.1016/j.rssm.2019.100444 Baron H, Young LE. From principles to practice: methods to increase the transparency of research ethics in violent contexts. Political Science Research and Methods. 2022. https://doi.org/10.1017/psrm.2021.48 Humphreys, M. (2015). Reflections on the ethics of social experimentation. Journal of Globalization and Development, 6(1), 87-112. https://doi.org/10.1515/jgd-2014-0016
Day 2, 9:00-12:00	Using Covariates in Experimental Design and Analysis / Clustered and blocked designs / Heterogeneous treatment effects / Power calculations Using covariates in experimental design Using pre-treatment variables to improve precision in estimating treatment effects Covariate balance with coding example - what if balance tests fail? Introduction to power calculations and effect size considerations Clustered and blocked designs
	Required reading: Gerber, A. S., & Green, D. P. (2012). Field experiments: Design, analysis, and interpretation. Chapter 4 & 9.
Day 2, 13:00-16:00	Threats to Validity: Non-compliance, Spillovers, Attrition and Experimenter effects / Pre-registration Treatment Heterogeneity Non-Compliance in Field Experiments Types of non-compliance: never-takers, always-takers, defiers Attrition and missing data Causes of attrition, how it affects causal inference Transparency and pre-registration OSF and examples of pre-registration (https://www.cos.io/initiatives/prereg) Pitfalls of Implementing Field Experiments Critiques of Field Experiments and Replies
	Required reading: Gerber, A. S., & Green, D. P. (2012). Field experiments: Design, analysis, and interpretation. Chapter 5.1, 5.2, 5.3, 5.4, chapter 6.1, 6.2, chapter 7. Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114 Suggested reading: Kalla, J. L. and Broockman, D. E. (2020). Reducing Exclusionary Attitudes through Interpersonal Conversation: Evidence from Three Field Experiments. American Political Science Review, 114(2), 410-425 https://doi.org/10.1017/s0003055419000923 Gerber, A. S., & Green, D. P. (2012). Field experiments: Design, analysis, and interpretation. Chapters 5 & 6 & 8. Lo, A., Renshon, J., & Bassan-Nygate, L. (2024). A Practical Guide to Dealing with Attrition in Political Science Experiments. Journal of Experimental Political Science, 11(2), 147-161. Engzell, P., & Rohrer, J. M. (2021). Improving Social Science: Lessons from the Open Science Movement. PS: Political Science & Politics, 54(2), 297-300. https://doi.org/10.1017/S1049096520000967 Grady, C., & Chen, N. 10 Things to Know About Pre-Analysis Plans: https://egap.org/resource/10-things-to-know-about-pre-analysis-plans/

Wiss. Koordination

Administrative Koordination

Field Experiments

About the lecturer - Johanna Gereke

About the lecturer - Johanna Gereke

About the lecturer - Leonard Wendering (TA)

About the lecturer - Leonard Wendering (TA)

Course description

Target group

Learning objectives

Prerequisites

Schedule

Schedule

Recommended readings

Recommended readings