Power Analysis Through Simulation in R
13.11 - 14.11.2023
13.11 - 14.11.2023
Online via Zoom
Online via Zoom
Students: 200 €
Academics: 300 €
Commercial: 600 €
Lecturer(s): Niklas Johannes
Statistical power is the probability of finding an effect if there truly is one to find. Unfortunately, it is often treated as a hoop to jump through so you can appease the ethics board or a reviewer. That's a shame, because knowing what you power for means you know what your study is all about. Power analysis lets you calculate the sample size needed for your next study or the sensitivity of your already conducted study. Attending this workshop will give you a deeper understanding of what you're after in your empirical analyses, rather than following heuristics. What heuristics? Ideally, after this workshop, you can immediately see what's problematic about a sentence like this:
“Following previous work (random citation), we calculated power for a two-way ANOVA. Relying on a medium effect size, we needed 26 people per group for 80% power.”
Instead of heuristics, we'll learn how to use simulations for calculating statistical power - but far more important, you'll learn about the thinking behind a simulation to make all aspects of your study explicit and transparent. You'll have everything you need to run a power simulation in R after half a day. But hopefully, you'll come away from this workshop with a good understanding of why you simulated your power a certain way.
In terms of analysis, we won't go beyond (repeated measures) interactions (aka regressions and ANOVAs). We won't cover more complicated designs like multilevel models-although after visiting this workshop, you'll have all tools to do that if you want to.
Participants will find the course useful if:
- they design and/or use empirical analyses
- they want to get an understanding of what power and effect size mean
- they want to learn how to evaluate which effects can be detected with what sample size
- they analyze experimental and observational data (we'll go through exercises for both)
The goal of the workshop is for you to (1) have an understanding of the philosophy behind using data to test claims, (2) get an intuition of how data generation processes work, (3) learn the technical skills in R to turn these processes into data, and (4) use these skills to simulate power for an informative study.
Each of the following blocks has a several learning outcomes. They are:
- Understand the logic behind null hypothesis significance testing (NHST)
- Get an intuition about what power is
- See why power, perhaps, potentially isn't just a hoop to jump through
Simulations in R
- Understand why simulations are useful
- Learn about the? logic of Monte Carlo Simulations
- Learn basic commands in R for simulating data
- Understand the importance of effect sizes
- Learn how to formulate a smallest effect size of interest
- Know when you don't have enough information
Alpha, beta, sensitivity (optional)
- Question the default of α = 0.05 and power = 80%
- Understand how terribly complex designing an informative study is
- Know where to turn when you don't have enough information
- Understand the logic behind the data generating process
- See how the linear model is our data generating process
- Apply this to a setting with multiple categories in a predictor
- Understand what an interaction is from the perspective of the linear model
- Make yourself think in more detail about the form of interactions
- Be able to translate that detail to generating data
- Understand that continuous predictors are just another case of the linear model
- Extend this understanding to continuous (by categorical) interactions
- Be able to translate that extension to generating data
Organizational structure of the course
The workshop will follow a structure of learn, do, recall. So typically, for each topic, I'll introduce theoretical concepts; then you'll do lots of exercises where you try to apply and extend these concepts; afterwards, there will be a short quiz for you to check your understanding of the core learnings.
The workshop is hands-on with lectures not taking too long (around max. 45 minutes per session), so that we have enough time for exercises. Usually, I first ask you to do the exercises by yourself or in groups, then I check in on people, and aim to solve them together in plenum.
In an online setting, this might require more group work so that people can help each other. I'll share all course materials with you so that you can re-visit topics even after the workshop is over. During the course, we'll use Discord for communication (e.g., discussing exercises, sharing code).
R, RStudio, GPower, Discord