Meta·Data Scientist·Technical Phone Screen
Apr 2026
Meta DS interview focused entirely on one deep system design question about replacing a legacy ads ranking model. The whole thing was a multi-part beast covering offline eval, online experimentation, and post-launch monitoring. Felt more like a product/ML systems design loop than a typical DS screen.
- You have a legacy ads ranking model and a new challenger model built by another team. Walk through your complete plan to validate and replace the old model, from offline checks all the way to production rollout.
- How would you use offline logs from the old model to estimate how the new model would have performed, and what are the assumptions and limitations of that approach?
- Your experiment shows CTR dropped about 1.5% but revenue per thousand impressions went up 3% and advertiser return on ad spend improved. How do you decide whether to ship the new model?
- What does your ramp schedule look like for rolling out the new model, and what would trigger a rollback at each stage?
- What are the main production risks after the new model is fully launched, and how would you monitor for them?
“This question is massive and I underestimated how structured they wanted the answer.” The rest of the author's notes on Data Scientist interview at Meta, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post