Pinterest·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

Senior

Apr 2026

Summary

Pinterest system design question focused on recommender systems. Pretty technical and specific, the kind of thing you either know from having built it or you're doing math on the fly.

Questions Asked (1)

In a two-stage recommender system with retrieval and ranking, how do you decide how many candidates to pass from retrieval into the ranking model? Walk through the tradeoffs around inference cost, latency, and cluster capacity.

System DesignTechnical Trade-offs

Author's notes

This one required actual back-of-the-envelope thinking.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Frame the candidate count as a tunable hyperparameter that balances retrieval recall, ranking model capacity, and system constraints. Discuss how to set it empirically using offline metrics and online A/B tests, while considering latency and cost budgets. Emphasize that the optimal number depends on the ranking model's ability to handle noise and the retrieval stage's precision.

Pro tip: Mention that you can use a lightweight ranking model or a cascade of rankers to filter candidates progressively, reducing the load on the heaviest model. Also, highlight the importance of monitoring the recall@k of retrieval to ensure you're not missing relevant items.

1. Define the objective and constraints

Clarify the goal: maximize ranking quality (e.g., NDCG, CTR) subject to latency and cost budgets. Identify the available cluster capacity and the ranking model's inference cost per candidate.

2. Analyze retrieval recall and ranking capacity

Measure retrieval recall@k to understand how many relevant items are captured as k increases. Assess the ranking model's performance and inference time as a function of candidate set size.

3. Model the tradeoff quantitatively

Estimate the marginal gain in ranking quality per additional candidate versus the marginal increase in latency and cost. Use offline simulations or small-scale online experiments to find the sweet spot.

4. Consider system architecture and cascades

Evaluate using a multi-stage ranking cascade (e.g., lightweight ranker then heavy ranker) to handle more candidates without exceeding latency. Decide the number of candidates passed to each stage.

5. Validate and iterate with online metrics

Run A/B tests to measure the impact of different candidate counts on user engagement and system metrics. Continuously monitor and adjust as models and traffic evolve.

Key Points to Mention

Retrieval recall@k and its impact on ranking quality
Ranking model inference cost and latency scaling with candidate set size
Cluster capacity and cost constraints (CPU/GPU, memory, throughput)
Offline evaluation metrics (e.g., NDCG, MRR) and online A/B testing
Cascading rankers or multi-stage ranking to balance quality and efficiency
Dynamic candidate count based on query or user context

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.