← CVS Health Interview Insights
The pooled proportion part tripped me up slightly.
First, compute the pooled proportion and the standard error of the difference. Then calculate the z-statistic and find the two-sided p-value using the standard normal distribution. Clearly state the formulas and assumptions.
Pro tip: Always check the assumptions of the two-proportion z-test, such as independent samples and sufficient sample size (at least 10 successes and failures per group). Also, mention that the p-value alone doesn't indicate practical significance; consider the effect size and confidence interval.
Compute the conversion rate for control (520/10000 = 0.052) and variant (630/11500 ≈ 0.05478). Note the difference in rates.
Pool the successes and trials: (520+630)/(10000+11500) = 1150/21500 ≈ 0.05349. This is the pooled proportion under the null hypothesis.
Compute the standard error of the difference: sqrt( pooled_prop * (1 - pooled_prop) * (1/n1 + 1/n2) ). With n1=10000, n2=11500, SE ≈ sqrt(0.05349*0.94651*(1/10000+1/11500)) ≈ 0.00302.
Calculate the z-statistic: (p1 - p2) / SE = (0.052 - 0.05478) / 0.00302 ≈ -0.92. The negative sign indicates control has lower rate, but for two-sided test we use absolute value.
For a two-sided test, p-value = 2 * P(Z < -|z|) = 2 * Φ(-0.92) ≈ 2 * 0.1788 = 0.3576. Since p > 0.05, we fail to reject the null hypothesis.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Switched from pooled to unpooled and had a brief moment of confusion about why you'd use unpooled for the CI but pooled for the test.
First, recall the formula for the confidence interval for the difference in proportions using unpooled standard error: (pB - pA) ± z* * sqrt( pA(1-pA)/nA + pB(1-pB)/nB ). Then, plug in the given conversion rates and sample sizes, compute the standard error, and find the critical z-value for 95% confidence (1.96). Finally, calculate the lower and upper bounds and interpret the interval in the context of the business question.
Pro tip: Always state the assumptions (e.g., independent samples, large enough sample sizes for normal approximation) and clarify that the unpooled SE is appropriate for confidence intervals (unlike hypothesis tests where pooled SE is used). This shows you understand the nuance and avoids common mistakes.
Extract the conversion rates (pA, pB) and sample sizes (nA, nB) from the problem statement or data. Ensure you know which group is A and which is B.
Calculate the unpooled standard error using the formula: SE = sqrt( pA(1-pA)/nA + pB(1-pB)/nB ).
For a 95% confidence interval, use the z-score corresponding to 97.5% percentile (1.96 for large samples). If sample sizes are small, consider using t-distribution, but typically z is used for proportions.
Compute the margin of error: ME = z * SE. Then the confidence interval is (pB - pA) ± ME. Report the lower and upper bounds.
Explain what the interval means in context: e.g., 'We are 95% confident that the true difference in conversion rates (B - A) lies between X and Y.' Also note if the interval includes zero, indicating no significant difference at 5% level.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Sample size planning is one of those things I've done a hundred times but always have to think through carefully under pressure.
First, clarify that the baseline conversion rate (p1) is needed; if not provided, state a reasonable assumption. Then, use the standard formula for sample size per arm for two proportions: n = (Z_{α/2} + Z_β)^2 * (p1(1-p1) + p2(1-p2)) / (p2 - p1)^2, where p2 = p1 + 0.007. Plug in Z_{0.025}=1.96 and Z_{0.20}=0.84, compute n, and round up to the next integer.
Pro tip: Always mention that this is an approximation assuming normal approximation and equal allocation; for small baseline rates or lifts, consider exact methods or simulations. Also, note that real-world sample sizes may need inflation for non-compliance or multiple comparisons.
Confirm the baseline conversion rate (p1) and the absolute lift (0.007). If p1 is not given, state an assumption (e.g., 5% or 10%) and proceed.
Write the sample size formula for two proportions: n = (Z_{α/2} + Z_β)^2 * [p1(1-p1) + p2(1-p2)] / (p2 - p1)^2.
Substitute Z_{α/2}=1.96, Z_β=0.84, p2 = p1 + 0.007, and compute the numerator and denominator.
Compute n and round up to the nearest whole number to ensure at least the required sample size per arm.
State the final sample size per arm and mention assumptions (e.g., normal approximation, equal allocation) and potential need for adjustments.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by stating the Bonferroni formula: adjusted alpha = target FWER / number of tests. Then compute 0.05 / 12 = 0.00417 (or approximately 0.0042). Finally, discuss the trade-off between controlling false positives and reducing power, and mention alternatives if appropriate.
Pro tip: Acknowledge that Bonferroni is conservative, especially with correlated metrics, and briefly mention that methods like Holm-Bonferroni or Benjamini-Hochberg might be more powerful while still controlling error rates. This shows depth beyond the basic calculation.
Clarify that the goal is to control the family-wise error rate (FWER) at 0.05 across 12 independent tests. Note that Bonferroni correction divides the target alpha by the number of tests.
Compute the adjusted per-metric significance threshold: α_adjusted = 0.05 / 12 = 0.004166... (approximately 0.0042).
Explain that each individual metric must have a p-value less than 0.0042 to be considered statistically significant after controlling for multiple comparisons.
Mention that Bonferroni is conservative and may reduce power, especially if metrics are correlated. Briefly note alternatives like Holm-Bonferroni or false discovery rate (FDR) control if appropriate.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.