This is the kind of question where you can talk yourself into a corner fast.
Acknowledge that the choice depends on the estimand and data properties: with skewed, zero-inflated integer counts, the t-test's normality assumption is violated, so either Mann-Whitney U (for stochastic dominance) or a GLM (e.g., negative binomial or zero-inflated model) is more appropriate. Recommend a GLM-based approach if you need to model the mean and adjust for covariates, but mention Mann-Whitney as a robust non-parametric alternative for simple comparisons.
Pro tip: At Meta, interviewers value practical impact: emphasize that you'd first check if the difference is practically significant (e.g., effect size) and consider bootstrapping for confidence intervals, as it handles zero-inflation and skew well without strong assumptions.
Determine whether you're comparing means, medians, or overall distributions, and whether you need to adjust for covariates. This drives the choice of test.
For t-tests: normality and homoscedasticity (Welch's relaxes equal variance). For Mann-Whitney: independent samples and similar distribution shapes (except shift). For GLM: correct distributional family and link function.
Run descriptive statistics, histograms, QQ plots, and tests for zero-inflation (e.g., proportion of zeros). Check for overdispersion if considering Poisson GLM.
If the goal is a simple comparison of central tendency and assumptions fail, use Mann-Whitney U. If modeling the mean with covariates or needing interpretable coefficients, use a GLM (e.g., negative binomial or zero-inflated).
Run the chosen test, check diagnostics (e.g., residual plots for GLM), and compute effect sizes or confidence intervals (e.g., via bootstrap) to assess practical significance.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, set up the equations for the mean and variance using the given parameterization, then solve for r and p by dividing the variance by the mean to isolate a function of p. After finding p, substitute back to find r, and finally compute P(X=0) using the negative binomial probability mass function.
Pro tip: When solving, remember that the variance-to-mean ratio (6.96/2.40 = 2.9) is greater than 1, indicating overdispersion, which is typical for count data like daily sessions. Also, double-check your algebra because small errors in p can significantly affect P(X=0).
Write the given mean and variance formulas: E[X] = r(1-p)/p = 2.40 and Var[X] = r(1-p)/p^2 = 6.96.
Divide the variance equation by the mean equation to eliminate r, yielding (1-p)/p = 6.96/2.40 = 2.9. Solve for p: 1-p = 2.9p => 1 = 3.9p => p = 1/3.9 ≈ 0.2564.
Substitute p back into the mean equation: r(1-p)/p = 2.40. With p ≈ 0.2564, 1-p ≈ 0.7436, so r ≈ 2.40 * 0.2564 / 0.7436 ≈ 0.827.
Use the negative binomial PMF: P(X=0) = p^r (since (1-p)^0 = 1). Calculate p^r ≈ (0.2564)^0.827 ≈ exp(0.827 * ln(0.2564)) ≈ exp(0.827 * (-1.361)) ≈ exp(-1.125) ≈ 0.325.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Straightforward once you remember the SE for a difference in means is just the square root of the sum of the two sample variances divided by their respective n's.
Treat the difference in means as approximately normal with variance equal to the sum of the two cohort variances divided by their sample sizes. Compute the standard error, then construct the 95% CI as the difference in means ± 1.96 * SE. State any assumptions, such as independence and large-sample normality.
Pro tip: Mention that with 5,000 users per cohort, the normal approximation is well-justified, and if the metric is a count (sessions), a Poisson or negative binomial GLM might be more appropriate—but the delta method gives a quick, reasonable interval. Also, clarify whether the variance is for individual users or the mean; if it's the variance of individual sessions, the standard error uses that variance divided by n.
Note the mean (1.85) and variance (4.20) for new users, and the sample size per cohort (5,000). Clarify that these are for one cohort; the other cohort's mean and variance are not provided, so assume they are the same or that the difference is between two independent cohorts with these parameters.
Under independence, the variance of the difference in means is the sum of the variances of each mean: Var(mean) = variance / n. So SE = sqrt(4.20/5000 + 4.20/5000) = sqrt(2 * 4.20/5000) ≈ sqrt(0.00168) ≈ 0.041.
For a 95% confidence interval, use the z-score 1.96 (or 2 for approximation) since the sample size is large.
The difference in means is 0 (if comparing identical cohorts) or some value if specified. The 95% CI is difference ± 1.96 * SE. If difference is 0, the interval is approximately (-0.080, 0.080).
Mention that this uses the delta method / normal approximation, assumes independence between cohorts, and that the variance is known or well-estimated. Also note that if the metric is a count, a GLM approach might yield a slightly different interval.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
The p-value vs effect size piece is almost a gimme but d = 0.08 is genuinely tiny so the practical significance answer is pretty clear: statistically detectable, operationally irrelevant unless you're at massive scale.
First, explain that statistical significance (p < 0.05) indicates a likely non-zero effect, but practical significance depends on the effect size and business context. With Cohen's d = 0.08, the effect is tiny and likely not practically meaningful. Then, address multiple testing by applying a correction like Bonferroni or Benjamini-Hochberg, and finally, for count data, suggest using the ratio of means (or its log) and obtaining a confidence interval via bootstrap or the delta method.
Pro tip: At Meta, always tie statistical findings to business impact—e.g., even a tiny effect can matter at scale if the metric is revenue, but often it's noise. Also, mention that multiple testing corrections should be pre-registered to avoid p-hacking.
Explain that p = 0.04 means the result is statistically significant at α = 0.05, but Cohen's d = 0.08 is a very small effect (negligible). Practical significance requires considering the metric's sensitivity and business impact.
Discuss that a small effect might be practically significant if the metric is critical (e.g., revenue per user) and the sample size is large, but often it's not worth acting on. Recommend looking at confidence intervals and raw differences.
With 5 country segments, you're conducting multiple comparisons, inflating Type I error. Apply a correction like Bonferroni (α/5 = 0.01) or Benjamini-Hochberg to control false discovery rate, and note that p = 0.04 would not survive Bonferroni.
For count data (e.g., clicks, purchases), use the ratio of means (or its log) as a robust effect size. It's interpretable as a multiplicative change and less sensitive to outliers than Cohen's d.
Use bootstrap resampling (non-parametric) or the delta method (parametric) to compute a CI for the ratio. Bootstrap is preferred for robustness with count data.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.