Tubi·Data Scientist·Technical Phone Screen
Apr 2026
Tubi data scientist interview with a pretty deep A/B testing question that covers sample size derivation, sensitivity analysis across like six different scenarios, and overdispersed count data. Felt more like a stats exam than a conversation.
- For an A/B test with baseline conversion of 8% and a relative MDE of 8% (so +0.64 percentage points), derive and compute the required per-arm sample size using a normal approximation. Show your z-score terms.
- Holding everything else constant from the baseline setup, how does required sample size change qualitatively and quantitatively when: the MDE is halved, alpha tightens to 0.01, power increases to 90%, allocation shifts to 75/25, there is user-level clustering with ICC=0.02 and average cluster size 5, and you add group-sequential monitoring with two equally spaced looks using O'Brien-Fleming boundaries?
- If the metric you're testing is overdispersed count data following a negative binomial distribution, how would you re-estimate the required sample size?
“The derivation itself isn't bad if you've done it before.” The rest of the author's notes on Data Scientist interview at Tubi, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post