CVS Health·Data Scientist·Technical Phone Screen
- Given a before/after payment accuracy experiment, choose an appropriate hypothesis test and formally state the null and alternative hypotheses.
- Compute the test statistic and two-sided p-value for the difference in proportions between treatment and control. Also explain when you'd use a pooled versus unpooled standard error and justify your choice here.
- Build a 95% confidence interval for the difference in proportions (treatment minus control) using the experiment data.
- Summarize the experimental results in a single sentence for a non-technical stakeholder, and address whether the improvement is practically significant.
- Using the normal approximation, calculate the per-group sample size needed to detect an absolute improvement from 96.0% to 96.2% accuracy with 80% power at a two-sided alpha of 0.05. Show the formula and plug in the Z-scores explicitly.
“Pretty standard setup once you recognize it's two proportions.”