← Meta Interview Insights

Meta·Software Engineer·Onsite - System Design / Architecture·Staff

StaffPrefer not to say
Jun 2026

Summary

System design round at Meta for a Research Engineer role, focused entirely on integrating LLMs into a large-scale recommendation stack. Dense, technical, and honestly one of the more interesting interviews I've had in this space.

Questions Asked (8)

Q1

Why would you place an LLM at the final re-ranking stage rather than earlier in the retrieval or coarse-ranking stages of a recommendation pipeline?

System DesignTechnical Trade-offs
Author's notes

This felt like the anchor question for everything else.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Frame the answer around the trade-off between computational cost and ranking quality, emphasizing that LLMs are expensive and slow but highly effective at capturing nuanced user-item interactions. Explain that placing LLMs at the final re-ranking stage allows you to apply their power to a small set of top candidates, balancing latency and relevance. Highlight that earlier stages prioritize recall and efficiency, while final re-ranking focuses on precision and personalization.

Pro tip: Quantify the trade-off: mention that LLMs can be 10-100x more expensive per item than traditional models, so applying them to thousands of candidates is infeasible, but re-ranking only the top 100-500 items yields significant quality gains with manageable latency.

1. Clarify the pipeline stages

Briefly describe the typical recommendation pipeline: retrieval (thousands to millions of items), coarse ranking (hundreds to thousands), and final re-ranking (tens to hundreds). This sets context for why stage placement matters.

2. Discuss LLM characteristics

Explain that LLMs excel at understanding complex semantics, user intent, and nuanced preferences but are computationally expensive and have high inference latency. They are not suitable for processing large candidate sets.

3. Analyze trade-offs of early placement

If placed earlier, LLMs would bottleneck the system due to latency and cost, and may not provide proportional gains because early stages prioritize recall over precision. Also, early stages often use simpler features that LLMs don't need to handle.

4. Argue for final re-ranking

At the final stage, the candidate set is small, so LLM inference cost is manageable. The LLM can focus on fine-grained ranking among highly relevant items, leveraging its strength in capturing subtle signals and improving top-line metrics like CTR or engagement.

5. Conclude with system design principles

Summarize that this placement follows the principle of applying expensive models where they add the most value: on a small, high-quality candidate set. It balances efficiency and effectiveness, a key consideration in large-scale systems.

Key Points to Mention

  • Computational cost and latency constraints of LLMs
  • Recall vs. precision trade-off in multi-stage ranking
  • Scalability: LLMs cannot handle millions of items in real-time
  • LLM's strength in capturing nuanced user-item interactions
  • Diminishing returns of applying complex models to noisy, broad candidate sets
  • Industry examples (e.g., YouTube, Amazon) where complex models are used in final ranking

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

How would you design a semantic ID scheme to tokenize items into a compact vocabulary of discrete codes for use with an LLM-based ranker?

System DesignData Modeling
Author's notes

Spent too long explaining residual quantization before getting to the actual design decisions.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the goal: to map each item to a short sequence of discrete tokens that capture semantic meaning, enabling the LLM to reason over items efficiently. Then outline a two-stage design: first learn item embeddings from content and interaction data, then quantize them into a compact codebook using techniques like RQ-VAE or hierarchical k-means. Finally, discuss how to integrate these codes into the LLM ranker and evaluate trade-offs.

Pro tip: Emphasize that the semantic IDs should be hierarchical and compositional, so that similar items share prefixes, allowing the LLM to generalize and perform arithmetic-like reasoning over codes. Also mention the importance of aligning the codebook size with the LLM's vocabulary capacity to avoid token dilution.

1. Clarify Requirements and Constraints

Define what 'semantic' means for the items (e.g., content, collaborative signals) and the constraints: vocabulary size, sequence length, latency, and integration with existing LLM ranker.

2. Learn Item Representations

Generate embeddings for each item using multimodal content (text, image, video) and user interaction data (e.g., co-views, clicks) via a two-tower model or graph neural network.

3. Quantize Embeddings into Discrete Codes

Apply a quantization method like Residual Quantization VAE (RQ-VAE) or hierarchical k-means to map continuous embeddings to a sequence of discrete codes from a compact codebook, ensuring codes are semantically meaningful and hierarchical.

4. Integrate with LLM Ranker

Design how the LLM consumes these codes: as special tokens in the input, or as part of a generative retrieval setup where the LLM predicts the next code. Consider adding code embeddings to the LLM's embedding table and fine-tuning.

5. Evaluate and Iterate

Measure ranking performance (e.g., NDCG, recall) and codebook utilization. Analyze if codes capture semantics by checking nearest neighbors and prefix sharing. Iterate on quantization and LLM integration.

Key Points to Mention

  • Choice of quantization method (RQ-VAE, hierarchical k-means) and why it produces compact, semantic codes.
  • Handling of cold-start items by using content-based embeddings before interaction data is available.
  • Trade-offs between codebook size, sequence length, and LLM vocabulary capacity.
  • How to ensure codes are hierarchical and compositional for better generalization.
  • Integration strategies: adding codes as special tokens vs. using them in a generative retrieval framework.
  • Evaluation metrics and offline/online testing to validate the semantic ID scheme.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

How do you handle cold-start items that have no training history when using a semantic ID scheme?

System DesignTechnical Trade-offs
Author's notes

Short answer: map new items to existing semantic codes based on content features at index time, before any interaction data exists.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the semantic ID scheme and the cold-start scenario, then discuss a hybrid approach that combines content-based features with fallback strategies. Emphasize the trade-offs between exploration and exploitation, and propose a concrete solution like using a separate embedding space or meta-learning.

Pro tip: Mention that cold-start is not just about new items but also new users and contexts; showing awareness of the full spectrum demonstrates depth. Also, highlight the importance of logging and feedback loops to quickly adapt as interaction data arrives.

1. Clarify the problem

Define what 'cold-start' means in this context: items with no interaction history, and how semantic IDs are generated (e.g., from content features).

2. Leverage content features

Use item metadata (text, images, categories) to create a content-based embedding that can be mapped to the semantic ID space or used as a fallback.

3. Design a hybrid retrieval strategy

Combine semantic ID-based retrieval with content-based retrieval, and use a gating mechanism to decide which to use based on item popularity or confidence.

4. Incorporate exploration

Allocate some traffic to explore cold items, using techniques like epsilon-greedy or Thompson sampling, to gather interaction data quickly.

5. Monitor and adapt

Set up logging and a feedback loop to update embeddings and semantic IDs as data accumulates, and evaluate performance metrics for cold items.

Key Points to Mention

  • Content-based embeddings as a fallback for cold items
  • Hybrid retrieval combining semantic IDs and content features
  • Exploration-exploitation trade-off (e.g., epsilon-greedy, bandits)
  • Meta-learning or few-shot learning to generalize from few examples
  • Importance of logging and fast feedback loops for online learning
  • Evaluation metrics for cold-start performance (e.g., coverage, CTR)

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

What input and output format would you design for the LLM in this re-ranking stage, and would you use a sequence-to-sequence approach or a scoring head?

System DesignTechnical Trade-offs
Author's notes

I went back and forth on this live and it probably showed.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the re-ranking task: it takes a query and a list of candidate documents and outputs a relevance score or ranking. Then compare sequence-to-sequence (generative) and scoring head (discriminative) approaches, focusing on efficiency, latency, and training data requirements. Recommend a scoring head for most re-ranking scenarios due to its simplicity and speed, but mention seq2seq when interpretability or listwise generation is needed.

Pro tip: Emphasize that re-ranking is latency-sensitive and typically operates on a small candidate set (e.g., top-100), so a scoring head with a cross-encoder architecture is often the pragmatic choice. Also, mention that you can use a seq2seq model to generate a permutation or relevance labels, but that adds complexity and inference cost.

1. Clarify the re-ranking task

Define the input as a query and a set of candidate documents (or passages) and the output as a relevance score per document or a ranked list. Mention that re-ranking is usually the second stage after retrieval.

2. Discuss input/output format options

For input, consider concatenating query and document with special tokens (e.g., [CLS] query [SEP] doc [SEP]). For output, a scalar score (pointwise), a pairwise preference, or a permutation (listwise).

3. Compare seq2seq vs. scoring head

Seq2seq: generative, can output a ranking or relevance text, but slower and harder to train. Scoring head: discriminative, adds a linear layer on top of a transformer to output a score, faster and simpler.

4. Evaluate trade-offs

Consider latency, training data, and interpretability. Scoring head is efficient for large-scale re-ranking; seq2seq may be useful for few-shot or when generating explanations.

5. Recommend a design

Propose a scoring head with a cross-encoder for most cases, and mention that seq2seq could be used if the task requires generating a ranked list or explanations.

Key Points to Mention

  • Cross-encoder architecture for scoring head
  • Pointwise, pairwise, and listwise ranking approaches
  • Latency and computational cost of seq2seq vs. scoring head
  • Training data requirements (e.g., labeled relevance judgments)
  • Use of special tokens to separate query and document
  • Possibility of using a seq2seq model for listwise ranking or explanation generation

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

Given the strict latency requirements at the final ranking stage, what techniques would you use to make LLM inference fast enough?

System DesignTechnical Trade-offs
Author's notes

Quantization, distillation into a smaller model, and KV-cache reuse across the candidate set.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the latency budget and the scale of the final ranking stage, then propose a layered optimization strategy that combines model-level, system-level, and algorithmic techniques. Emphasize trade-offs between latency, accuracy, and cost, and give concrete examples of how you would measure and iterate.

Pro tip: Mention that you would profile the entire inference pipeline to identify the actual bottleneck, because often the LLM is not the only latency contributor—feature fetching and network overhead can dominate. Also, highlight that you would consider caching and precomputation for repeated queries, which is often overlooked.

1. Clarify requirements and constraints

Ask about the exact latency SLA (e.g., p99 < 100ms), throughput, and whether the LLM is used for scoring or generation. Understand the hardware and deployment environment.

2. Optimize the model

Discuss model compression techniques like quantization (INT8, FP16), pruning, and distillation to a smaller model. Consider using a smaller LLM or a specialized architecture for ranking.

3. Optimize inference runtime

Leverage optimized inference engines (TensorRT, ONNX Runtime, vLLM), kernel fusion, and batch processing. Use techniques like speculative decoding or continuous batching to increase throughput.

4. Optimize system and infrastructure

Deploy on GPUs/TPUs with high memory bandwidth, use caching (e.g., KV cache, result caching), and precompute embeddings or features. Consider model parallelism or sharding for large models.

5. Measure, iterate, and trade off

Profile end-to-end latency, identify bottlenecks, and iterate. Be prepared to trade off accuracy for latency (e.g., approximate nearest neighbors, early exit).

Key Points to Mention

  • Quantization and pruning to reduce model size and compute
  • Use of optimized inference engines like TensorRT, ONNX Runtime, or vLLM
  • Caching strategies: KV cache, result caching, and precomputed embeddings
  • Batching and continuous batching to improve GPU utilization
  • Model distillation to a smaller, faster model
  • Hardware acceleration (GPU/TPU) and memory bandwidth considerations
  • Trade-offs between latency, accuracy, and cost

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q6

How would you train this LLM ranker, and what role could reinforcement learning from human feedback play using engagement as the reward signal?

System DesignTechnical Trade-offs
Author's notes

Started with supervised fine-tuning on logged exposure and conversion data, which is the obvious first step.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by outlining the full training pipeline for an LLM ranker, from pretraining to supervised fine-tuning to reinforcement learning. Then focus on RLHF, explaining how engagement signals like clicks and dwell time can be used as rewards, while addressing challenges like reward hacking and bias. Emphasize evaluation and iteration to ensure alignment with business goals.

Pro tip: Acknowledge that engagement metrics are noisy and can be gamed; propose using a combination of offline metrics and online A/B testing, and consider techniques like reward shaping or constrained RL to mitigate unintended consequences.

1. Define the ranking task and data

Clarify the input (query, user context, documents) and output (ranked list). Discuss data sources: user interactions (clicks, dwell time, conversions) and relevance labels.

2. Outline the training pipeline

Describe stages: pretraining on large text corpora, supervised fine-tuning on ranking data (e.g., pairwise or listwise losses), and optional reinforcement learning from human feedback.

3. Explain RLHF with engagement rewards

Detail how to collect human feedback (e.g., pairwise comparisons) to train a reward model, then use RL (e.g., PPO) to optimize the ranker against that reward. Discuss using engagement signals as implicit rewards.

4. Address challenges and trade-offs

Discuss issues like reward hacking, position bias, and delayed feedback. Propose solutions: reward shaping, off-policy correction, and multi-objective optimization.

5. Evaluate and iterate

Describe offline evaluation (NDCG, MRR) and online A/B testing. Emphasize monitoring for unintended consequences and continuous improvement.

Key Points to Mention

  • Pretraining and fine-tuning stages for LLM rankers
  • Reward modeling from human feedback (pairwise comparisons)
  • Using engagement metrics (clicks, dwell time) as reward signals
  • Challenges: reward hacking, position bias, delayed feedback
  • Mitigation strategies: reward shaping, off-policy correction, constrained RL
  • Evaluation: offline metrics and online A/B testing

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q7

Describe the online serving architecture for this LLM-based ranker, including fallback paths and how you'd run an A/B test to evaluate it.

System DesignA/B Testing & Experimentation
Author's notes

Fallback to a classical ranker if the LLM times out or returns an error, with a circuit breaker pattern.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the requirements and constraints of the online serving system, then describe the end-to-end architecture from request to response, including the LLM ranker and fallback paths. Finally, outline a rigorous A/B testing plan to measure the ranker's impact on key metrics.

Pro tip: Emphasize the importance of monitoring and gradual rollout to mitigate risks, and discuss how you'd handle potential latency and cost issues with LLM inference in production.

1. Clarify Requirements and Constraints

Ask questions to understand scale (QPS, latency SLOs), cost constraints, and the current ranking system. Confirm the goal is to improve ranking quality with an LLM.

2. Design the Serving Architecture

Describe the components: request routing, feature fetching, LLM inference service, caching, and response assembly. Explain how the LLM ranker fits into the existing pipeline.

3. Define Fallback Paths

Outline fallback strategies for LLM failures or high latency, such as falling back to a traditional ranker or a simpler model. Discuss timeouts, circuit breakers, and graceful degradation.

4. Plan the A/B Test

Detail the experiment design: randomization unit, control vs. treatment, key metrics (e.g., CTR, engagement), sample size, and duration. Mention guardrail metrics and statistical analysis.

5. Address Monitoring and Iteration

Explain how you'd monitor system health and experiment results, and how you'd iterate based on findings. Include logging, alerting, and rollback procedures.

Key Points to Mention

  • Latency and cost optimization for LLM inference (e.g., model distillation, caching, batching)
  • Fallback mechanisms: timeouts, circuit breakers, and alternative rankers
  • A/B testing best practices: randomization, sample size calculation, and avoiding pitfalls like novelty effects
  • Metrics: primary (e.g., CTR, conversion) and guardrail (e.g., latency, error rates)
  • Scalability and reliability: load balancing, auto-scaling, and fault tolerance
  • Monitoring and observability: logging, tracing, and alerting for both system and experiment

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q8

What are the main tradeoffs between using an LLM-based ranker versus classical approaches like gradient-boosted trees or mixture-of-experts models?

Technical Trade-offsSystem Design
Author's notes

LLMs win on semantic understanding and the ability to reason over item sequences, but lose on latency, cost, and interpretability.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by acknowledging that the choice depends on the specific ranking task, data characteristics, and production constraints. Then compare LLM-based rankers and classical approaches across key dimensions like accuracy, latency, cost, interpretability, and scalability, using concrete examples. Conclude with a balanced view on when to use each or how to combine them.

Pro tip: Emphasize that LLMs excel at semantic understanding and zero-shot generalization but struggle with latency and cost, while classical models are efficient and interpretable but require feature engineering. Mention hybrid approaches, such as using LLMs for feature generation or re-ranking, to show depth.

1. Define the ranking problem

Clarify the specific ranking scenario (e.g., search, recommendation, ads) and the key metrics (e.g., NDCG, CTR, latency). This sets the context for tradeoffs.

2. Compare accuracy and generalization

Discuss how LLMs can capture complex semantics and generalize zero-shot, while classical models like GBDTs or MoE may require extensive feature engineering but can be highly accurate with sufficient data.

3. Evaluate latency and cost

Highlight that LLM inference is typically slower and more expensive, making it challenging for real-time ranking, whereas classical models are optimized for low-latency, high-throughput serving.

4. Consider interpretability and maintainability

Note that classical models offer better interpretability and easier debugging, while LLMs are black boxes and harder to update or control.

5. Discuss hybrid and deployment strategies

Propose hybrid approaches, such as using LLMs for offline feature generation or as a re-ranker on a small candidate set, to balance tradeoffs.

Key Points to Mention

  • LLMs provide strong semantic understanding and zero-shot capabilities but incur high inference latency and cost.
  • Classical models like GBDTs are efficient, interpretable, and well-suited for large-scale real-time ranking.
  • Mixture-of-experts models can scale model capacity while maintaining reasonable inference costs through conditional computation.
  • Data requirements: LLMs may need less task-specific data, while classical models require labeled data and feature engineering.
  • Hybrid systems: using LLMs for candidate generation or feature enrichment, and classical models for final ranking.
  • Production constraints: latency SLAs, cost per query, and infrastructure availability heavily influence the choice.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.