Any data analysis is based on a large number of decisions. These decisions relate to, among other things, study design, data preparation, and the selection and specification of statistical models (Rijnhart et al. 2021). Therefore, a single analysis represents only one possibility among a larger set of alternatives. This leads to the question of how much the analysis results depend on the often undocumented choices.

The relatively new approach of multiverse analysis (Steegen et al. 2016) addresses two fundamental problems in research: the lack of transparency and the dependence of analysis results on data-analytic decisions (Young 2018). The idea of multiverse analysis is to conduct not just one, but ideally all (meaningful) analyses and present the results in summary form. This can make the impact of data analytic decisions on the results transparent and assess whether the conclusions are robust to alternative (modeling) decisions.

However, multiverse analysis can be time-consuming and resource-intensive, and it introduces new questions and challenges. This refers especially to the comparison of the (many) results. On the other hand, it also makes aspects of the scientific process easier, as it relieves the researcher from the burden to "sell" the best possible story.

In the course, the basics of multiverse analysis are explained step by step, applied to a real data example in Stata, and the results are presented using a so-called specification curve (Simonsohn et al. 2020). Participants will gain practical experience in conducting a multiverse analysis. The focus is on the application of a multiverse analysis for collected data.