Roku·Data Scientist·Technical Phone Screen
- For a newly launched smart speaker with voluntary adoption, define a primary engagement metric and the guardrail metrics you'd put around it.
- Design two complementary strategies to estimate the causal effect of speaker adoption on engagement at 30 and 180 days, given you can't run a traditional A/B test.
- Use regional stockouts and shipping delays as an instrumental variable. Argue why it's a valid instrument and describe how you'd run falsification tests.
- Do a rough power calculation given 6% adoption rate and historical variance. What sample size or time horizon would you need to detect a 3% lift in engagement?
“I went with something like daily active streaming minutes per device, which felt right, but I fumbled the guardrail framing.”