Pinterest·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

Senior

Jun 2026

Summary

Pinterest ML engineer interview focused entirely on a notification system design problem. It was a deep, multi-part question that covered basically the full ML lifecycle, from objective setting to serving infrastructure. Pretty intense for a single session.

Questions Asked (1)

Design an ML system that decides which push or email notifications to send to each Pinterest user, when to send them, and how often, with the goal of maximizing long-term engagement without hurting retention.

System DesignProduct Analytics & MetricsA/B Testing & Experimentation

Author's notes

This thing had like six sub-problems inside it and I didn't pace myself well.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the business objectives and constraints, then propose a multi-stage ML system that includes candidate generation, ranking, and timing optimization. Emphasize the use of long-term metrics and experimentation to balance engagement and retention.

Pro tip: Highlight the importance of causal inference and counterfactual reasoning to avoid confounding from observational data, and propose a reinforcement learning approach with safety constraints to optimize long-term rewards.

1. Clarify Objectives and Constraints

Define what 'long-term engagement' and 'retention' mean, and identify constraints such as user fatigue, notification frequency limits, and privacy concerns.

2. Data and Feature Engineering

Collect user interaction data, content features, and contextual signals. Engineer features for user preferences, notification responsiveness, and temporal patterns.

3. Modeling Approach

Design a multi-stage system: candidate generation (which notifications), ranking (relevance), and timing/frequency optimization. Use models like gradient boosted trees for ranking and reinforcement learning for long-term optimization.

4. Evaluation and Metrics

Define offline metrics (e.g., AUC, NDCG) and online metrics (CTR, engagement, retention). Use A/B tests and long-term holdout groups to measure impact on retention.

5. Deployment and Iteration

Deploy with a feedback loop, monitor for drift, and continuously refine models. Incorporate user feedback and guardrails to prevent over-notification.

Key Points to Mention

Multi-stage architecture: candidate generation, ranking, and timing optimization
Use of reinforcement learning or bandits for long-term reward optimization
Importance of causal inference to measure true impact on retention
A/B testing framework with long-term holdout to detect retention effects
User fatigue and frequency capping mechanisms
Personalization based on user behavior and content affinity

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.