Pinterest·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

Pinterest ML Engineer system design round, focused entirely on embedding infrastructure at scale. The question was meaty and covered a lot of ground, from training paradigms to cold-start handling to ANN indexing. Left feeling like I did okay on the core architecture but probably undersold the evaluation side.

Questions Asked (1)

Design a system that learns and serves embeddings for users and items, to be used as input for downstream personalization tasks like recommendations, ranking, search, and ads.

System DesignTechnical Trade-offsA/B Testing & Experimentation

Author's notes

This one sprawls fast if you let it.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the scope and requirements, then design a two-part system: an offline training pipeline that learns embeddings from user-item interactions and content features, and an online serving layer that provides low-latency access to these embeddings for multiple downstream tasks. Emphasize how you would evaluate the embeddings both intrinsically and through online A/B tests on downstream metrics.

Pro tip: Highlight the importance of a shared embedding space across tasks to enable transfer learning and reduce redundancy, and discuss how you would handle cold-start users/items by incorporating content features and using fallback strategies.

1. Clarify Requirements and Scope

Ask questions to understand scale (number of users/items), latency requirements, update frequency, and which downstream tasks are prioritized. Clarify if embeddings need to be interpretable or just effective.

2. Design Offline Training Pipeline

Choose an embedding learning approach (e.g., matrix factorization, two-tower neural networks, graph-based) that leverages user-item interactions and side features. Describe data preprocessing, feature engineering, and training infrastructure (e.g., distributed training).

3. Design Online Serving Architecture

Outline how embeddings are stored (e.g., key-value store, vector database) and served with low latency. Discuss caching, sharding, and how to handle updates (batch vs. real-time).

4. Address Integration with Downstream Tasks

Explain how embeddings are consumed by recommendation, ranking, search, and ads models. Discuss whether to fine-tune embeddings per task or use them as fixed features, and how to handle multiple tasks efficiently.

5. Define Evaluation and Iteration Strategy

Propose offline evaluation metrics (e.g., recall@k, NDCG) and online A/B testing methodology. Describe how to monitor embedding quality and retrain/update embeddings over time.

Key Points to Mention

Choice of embedding model (e.g., two-tower, matrix factorization, graph neural networks) and rationale based on data availability and task requirements.
Handling cold-start users/items by incorporating content features and using fallback strategies like popularity-based embeddings.
Scalability considerations: distributed training, efficient storage (e.g., FAISS, Redis), and low-latency serving with caching.
Multi-task learning: sharing embeddings across tasks to improve generalization and reduce computational cost.
Evaluation metrics: offline (e.g., recall@k, NDCG) and online (A/B tests on CTR, engagement) to measure embedding effectiveness.
Update frequency: batch vs. online updates, and how to handle concept drift and freshness of embeddings.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.