Meta·Data Scientist·Technical Phone Screen
Jun 2026
A deep technical screen for a data scientist role at Meta, centered entirely on one massive experiment design question about a spam detection feature for Messenger. The level of detail expected was genuinely surprising, covering everything from cluster randomization math to launch ramp plans.
- For a new spam-detection algorithm on Messenger that routes suspected spam to a separate folder and delays delivery, define your primary success metric and at least three guardrail metrics. Be specific about denominators and attribution windows.
- Compare individual-user randomization versus cluster randomization by conversation thread, by recipient, and by geography for this experiment. For each, walk through how interference could occur and when SUTVA breaks down.
- What are the specific pitfalls of cluster randomization designs, and how would you mitigate them in this context?
- Given a baseline spam reply rate of 2%, a target of detecting a 10% relative reduction, alpha 0.05, power 0.8, and a 7-day test window with 200M suspected-spam messages per day: compute the design effect and effective sample size for thread-level clustering with average 3 messages per thread and ICC of 0.07. Then repeat for recipient-level clustering with 20 messages per recipient and ICC of 0.02. Which design do you recommend?
- Describe your full analysis plan: which estimator, how you handle inference, whether you'd use CUPED, and how you'd deal with noncompliance or multiple exposure types.
- Given effect sizes in the range of minus 8 to minus 12 percent relative reduction on spam reply rate with guardrails holding, what is your quantitative launch criterion and what does your ramp plan look like?
“Went with spam reply rate as the primary metric, which felt right, probability a recipient replies within 24 hours to a flagged message.” The rest of the author's notes on Data Scientist interview at Meta, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post