Dropbox·Data Scientist·Technical Phone Screen
- Design an online experiment to evaluate a file recommender system in production across multiple organizations, where users within the same org can influence each other's behavior. Cover: primary metrics, secondary metrics, guardrails, randomization unit, clustering and ICC assumptions for sample size, variance reduction, sequential monitoring, ramp schedule, novelty/carryover controls, logging for exposure and attribution, a difference-in-differences fallback if full randomization isn't possible, and stop/ship criteria including how to handle Simpson's paradox across tenants and roles.
“This was a lot.”