Pinterest·Data Scientist·Technical Phone Screen
Jun 2026
Pinterest DS interview with a single massive experiment design question covering metric selection, experiment design, power analysis, and decision-making all rolled into one. The depth required was pretty intense for what I expected to be a standard A/B testing discussion.
- You're launching a new Shopping module embedded in the Pins feed. Design a full experiment and metric plan: choose a primary success metric and 3-5 guardrail metrics with precise definitions (numerator, denominator, unit of analysis, aggregation window), including at least one funnel metric. Discuss the tradeoffs of DAU and time spent as primaries versus alternatives like Shopping CTR, Add-to-Cart Rate, or GMV per user.
- How would you handle spillover and interference in this experiment, given that repins and shares could expose control users to the Shopping module? Propose and justify a concrete randomization design, covering the randomization unit, eligibility rules, cooldown period, novelty burn-in, and how you'd detect and quantify spillover and learning effects.
- Walk through your power and MDE calculations for this experiment: what baseline rate assumptions would you make, how do you account for intra-cluster correlation if clustered, how long would you run it, and how would you handle weekly seasonality and peaky traffic? Also describe your AA test plan and any covariate adjustment strategy.
- After 21 days you see +2.3% in Shopping CTR and +1.1% in GMV per user, but -0.6% in overall time spent and -0.2% in DAU. How do you build a decision framework for this result? Define your guardrail thresholds, describe how you'd weight these metrics against each other, account for potential long-term effects, and give a concrete recommendation to the PM. What additional diagnostics would you run before any rollout decision?
“The part that tripped me up was being asked to define metrics with numerator/denominator precision rather than just naming them.” The rest of the author's notes on Data Scientist interview at Pinterest, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post