I knew the definition but fumbled when trying to connect all the moving pieces together.
Start with a clear definition of statistical power as the probability of correctly rejecting a false null hypothesis, then explain the four key drivers: effect size, sample size, significance level, and variability. Use a concrete A/B testing example to illustrate how these factors interact in practice.
Pro tip: Emphasize that power is not just a theoretical concept but a practical tool for experiment design—mention how Google uses power analysis to avoid underpowered tests and ensure reliable product decisions.
State that power is the probability of detecting a true effect when it exists, i.e., 1 minus the Type II error rate (beta). Clarify that it's a pre-experiment design parameter, not a post-hoc metric.
List the four main factors: effect size (larger effects are easier to detect), sample size (more data increases power), significance level (alpha, typically 0.05), and variability (less noise means more power).
Describe how these factors are interrelated: for a fixed power, increasing sample size allows detection of smaller effects; lowering alpha reduces power; higher variability requires larger samples.
Connect to A/B testing: power determines the minimum sample size needed to detect a meaningful lift, and underpowered tests lead to false negatives and wasted resources.
Summarize that power analysis should be done before launching an experiment to ensure valid, actionable results, and mention tools like power calculators or simulations.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.