Meta·Data Scientist·Technical Phone Screen
- How would you define the objectives for an ad recommendation system in a content feed, and how would you balance revenue against user experience?
- What modeling approach would you use to predict CTR, CVR, and revenue for ads, and how would you handle delayed conversion feedback?
- How would you address selection bias when evaluating a new ranking policy if your training data came from a previous policy's logged decisions?
- How would you approach exploration vs. exploitation in an ad ranking system?
- How would you handle cold start for new ads and new users in your ranking system?
- How would you incorporate advertiser budget pacing constraints into the ranking and serving system?
- Walk me through your offline and online evaluation plan for this ad ranking system, including what guardrail metrics you would set.
“I started with the obvious revenue angle and then pivoted to user experience, but I fumbled the part about how to actually combine them.”