Scientific Coordination
Alisa Remizova
Administrative Coordination
Claudia O'Donovan-Bellante
Please wait...
Applied Multiverse Analysis with Stata and R
About
Location:
Online via Zoom
Online via Zoom
General Topics:
Course Level:
Format:
Software used:
Duration:
Language:
Fees:
Students: 330 €
Academics: 495 €
Commercial: 990 €
Keywords
Additional links
Lecturer(s): Maximilian Brinkmann, Johanna Pauliks, Reinhard Schunck
Course description
Any data analysis is based on a large number of decisions. These decisions relate to, among other things, study design, data preparation, and selection and specification of statistical models (Rijnhart et al. 2021). Therefore, a single analysis represents only one possibility among a larger set of alternatives. This leads to the question of how much the analysis results depend on the often undocumented choices.
The relatively new approach of multiverse analysis (Steegen et al. 2016) addresses two fundamental problems in research: the lack of transparency and the dependence of analysis results on data-analytic decisions (Young 2018). The idea of multiverse analysis is to conduct not just one but ideally all (meaningful) analyses and present the results in summary form. This can make the impact of data analytic decisions on the results transparent and assess whether the conclusions are robust to alternative (modeling) decisions.
However, multiverse analysis can be time-consuming and resource-intensive, and it introduces new questions and challenges. This refers especially to the comparison of the (many) results. On the other hand, it also makes aspects of the scientific process easier, as it relieves the researcher from the burden of "selling" the best possible story.
In the course, the basics of multiverse analysis are explained step by step, applied to a real data example in Stata and R, and the results are presented using a so-called specification curve (Simonsohn et al. 2020). You will gain practical experience in conducting a multiverse analysis. The focus is on the application of a multiverse analysis to the collected data. The course is "bilingual", with computer code to support R and Stata users.
Organizational structure of the course
The course will feature about three hours of classroom instruction and three hours of hands-on exercises and/or group work each day.
Classroom instruction: The lecturers will provide an overview of different aspects of multiverse analysis, including the methodological background, the implementation in Stata or R, and practical challenges. The lecturers will also explain the tasks for the hands-on exercises and discuss the results of the exercises.
Individual exercises: In each exercise, you are expected to work on assignments in groups. The lecturers are available during the exercises to assist or to provide guidance.
If time permits, you may consult lecturers for more extensive help and guidance on your personal projects. If you are interested in individual consultations concerning your ongoing projects, you are encouraged to contact the lecturers before the course and to provide a short description of the issues you would like to discuss.
Target group
You will find the course useful if:
- you are interested in how to improve the reliability of inferences in quantitative research.
- you are interested in how to conduct research openly and transparently.
- you are interested in how to apply a multiverse analysis in Stata or R.
- you plan to conduct a multiverse analysis in your own research.
Learning objectives
By the end of the course, you will:
- learn and discuss the features, typical applications, advantages, and shortcomings of multiverse analysis.
- acquire practical insights into the steps that are needed to set up a multiverse analysis, to conduct them, and how to interpret their results.
- be able to conduct a multiverse analysis on your own in Stata or R.
Prerequisites
Software requirements
You need a laptop/desktop computer that enables you to access the internet and smoothly work with Stata or R. If needed, you will be provided with access to Stata licenses by GESIS but must install the software prior to the course on their own devices Please contact GESIS two weeks in advance of the course if you need a license.
Stata ados used during the course (and, ideally, installed before the course) include:
R packages used during the course (and, ideally, installed before the course) include:
