LinkedIn·Data Scientist·Technical Phone Screen
- A marketing team ran an A/B test on two email versions across two cities and two weeks. The pooled data shows version A winning, but version B wins in every city-level and week-level breakdown. How do you figure out which email is actually better? Walk through your choice of primary and guardrail metrics, how Simpson's paradox shows up here, what confounders you'd investigate, how you'd reanalyze the data, whether confidence intervals are still valid, and how you'd redesign the experiment if the original allocation was imbalanced.
“This one sprawled in a way I wasn't ready for.”