Uber·Data Scientist·Technical Phone Screen·Intermediate

Intermediate

Jul 2026

Summary

Uber data scientist interview, stats-heavy technical screen. The whole thing was basically one long hypothesis testing question with a few layers stacked on top of each other. Not a vibe-check round at all.

Questions Asked (1)

For an email marketing A/B test comparing click-through rates between a control and a test subject line, how would you set up the null and alternative hypotheses, justify using a z-test via the Central Limit Theorem, and derive the sample size needed to detect a 2 percentage point lift with 80% power at a 0.05 significance level?

A/B Testing & ExperimentationProduct Analytics & Metrics

Author's notes

This is three questions dressed up as one and I didn't realize that until I was halfway through answering the first part.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly defining the null and alternative hypotheses for the difference in click-through rates between control and test. Then justify the use of a z-test by invoking the Central Limit Theorem for large sample sizes, and finally derive the required sample size using the standard formula for two proportions, incorporating the given significance level and power.

Pro tip: Always consider practical significance alongside statistical significance; a 2 percentage point lift might be statistically significant but not business-relevant depending on baseline CTR and costs. Also, mention that you'd check assumptions like independence and random assignment before proceeding.

1. Define Hypotheses

Set up the null hypothesis (H0) that there is no difference in click-through rates between control and test, and the alternative hypothesis (H1) that there is a difference (two-tailed) or that test is greater (one-tailed). Specify the parameters: p_c for control and p_t for test.

2. Justify z-test via CLT

Explain that since click-through is a binary outcome, the sample proportion follows a binomial distribution. For large samples, the Central Limit Theorem ensures the sampling distribution of the difference in proportions is approximately normal, allowing the use of a z-test.

3. Derive Sample Size Formula

Use the standard formula for sample size per group for comparing two proportions: n = ( (z_{α/2} + z_β)^2 * (p1(1-p1) + p2(1-p2)) ) / (p2 - p1)^2, where p1 is baseline CTR, p2 = p1 + 0.02, α=0.05, power=0.80.

4. Plug in Values and Compute

Assume a baseline CTR (e.g., 10% or 20%) if not given, compute the required sample size per group, and round up to the nearest integer. Mention that if baseline is unknown, use a conservative estimate (e.g., 50%) or pilot data.

5. Interpret and Discuss Assumptions

State the final sample size and discuss assumptions: random assignment, independence, sufficient sample size for normality, and that the test is two-tailed. Also note that this is a simplified calculation and real-world factors like multiple testing or sequential analysis may require adjustments.

Key Points to Mention

Null hypothesis: H0: p_t - p_c = 0; Alternative: H1: p_t - p_c ≠ 0 (or > 0 for one-tailed).
Central Limit Theorem justifies normal approximation for large samples, enabling z-test.
Sample size formula for two proportions: n = ( (z_{α/2} + z_β)^2 * (p1(1-p1) + p2(1-p2)) ) / (p2 - p1)^2.
Use z_{α/2} = 1.96 for 95% confidence, z_β = 0.84 for 80% power.
Baseline CTR assumption is critical; if unknown, use 50% for maximum variance or historical data.
Consider practical significance and business impact beyond statistical significance.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.