← Openai Interview Insights

Openai·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
Apr 2026

Summary

Brutal system design round at OpenAI for an MLE role. The question was essentially a full RAG architecture design from scratch, covering everything from ingestion to A/B testing to capacity math. Felt like they were asking for a 2-hour design doc in 45 minutes.

Questions Asked (6)

Q1

Design a production-grade RAG system for a customer support assistant that meets strict p99 latency under 1.5 seconds and has real cost constraints. Walk through the full architecture: document ingestion, chunking, embedding model choice, index type, caching, re-ranking, prompt orchestration, and safety guardrails.

System DesignTechnical Trade-offs
Author's notes

This is basically a full system design in one breath.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then walk through the architecture end-to-end, making explicit trade-offs at each stage to meet p99 latency and cost goals. Emphasize how each component (ingestion, retrieval, re-ranking, caching, orchestration, guardrails) contributes to the overall performance and cost profile, and justify your choices with data-driven reasoning.

Pro tip: Quantify the impact of each design decision on latency and cost—for example, estimate the p99 latency budget per component and show how caching and re-ranking trade-offs keep you under 1.5s. Also, mention that you'd validate with load testing and continuous monitoring to ensure SLAs are met in production.

1. Clarify Requirements and Constraints

Ask about expected QPS, document volume, update frequency, and cost budget. Confirm that p99 latency <1.5s is end-to-end and identify any compliance or safety requirements.

2. Design Ingestion and Indexing Pipeline

Outline document ingestion (batch/streaming), chunking strategy (e.g., semantic or fixed-size with overlap), embedding model choice (e.g., text-embedding-3-small for cost/latency), and index type (e.g., HNSW for low-latency ANN search).

3. Optimize Retrieval and Re-ranking

Describe hybrid retrieval (dense + sparse) and re-ranking with a lightweight cross-encoder or LLM-based re-ranker, balancing accuracy and latency. Discuss caching strategies (e.g., query cache, embedding cache) to reduce redundant computation.

4. Orchestrate Prompt and Guardrails

Explain prompt construction with retrieved context, LLM selection (e.g., GPT-4o-mini for cost/latency), and safety guardrails (e.g., input/output moderation, PII redaction, fallback responses).

5. Validate and Monitor Performance

Propose load testing to measure p99 latency, cost tracking, and monitoring for drift. Discuss iterative improvements based on metrics.

Key Points to Mention

  • Latency budget breakdown: embedding, retrieval, re-ranking, LLM inference, and network overhead.
  • Cost optimization: using smaller embedding models, caching frequent queries, and batching.
  • Chunking strategy: balancing context length and retrieval granularity (e.g., 256-512 tokens with overlap).
  • Index choice: HNSW for fast approximate nearest neighbor search with tunable parameters (efSearch, M).
  • Re-ranking trade-off: using a lightweight model or skipping re-ranking if latency budget is tight.
  • Safety guardrails: moderation APIs, prompt injection defense, and fallback to human agent.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

How would you handle document updates and deletions in the vector index, and how do you enforce data isolation across multiple tenants?

System DesignData Modeling
Author's notes

The deletion problem is sneakier than it sounds.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the requirements: scale, latency, consistency, and tenant isolation guarantees. Then propose a design that handles updates/deletions via soft deletes with periodic compaction, and enforces tenant isolation through namespace partitioning and metadata filtering. Finally, discuss trade-offs and alternatives.

Pro tip: Emphasize the importance of measuring recall and latency after updates, and consider using a write-ahead log for durability. Also, mention that tenant isolation should be enforced at multiple layers (e.g., index, query, and access control) to prevent data leakage.

1. Clarify Requirements

Ask about scale (number of documents, tenants), update frequency, latency requirements, and consistency needs. This shapes the design choices.

2. Design for Updates and Deletions

Propose using soft deletes (tombstones) and versioning, with periodic compaction to reclaim space and maintain performance. Discuss how to handle updates efficiently without full reindexing.

3. Enforce Tenant Isolation

Suggest partitioning the index by tenant (e.g., separate namespaces or collections) and using metadata filtering to ensure queries only access the tenant's data. Mention access control at the API layer.

4. Address Trade-offs and Alternatives

Compare approaches: separate indices per tenant vs. shared index with filters. Discuss trade-offs in terms of cost, performance, and isolation guarantees.

5. Summarize and Validate

Recap the proposed solution, highlighting how it meets the requirements, and suggest metrics to monitor (e.g., recall, latency, isolation breaches).

Key Points to Mention

  • Soft deletes with tombstones and periodic compaction
  • Versioning of documents to handle updates
  • Namespace or collection per tenant for isolation
  • Metadata filtering with tenant ID in queries
  • Access control and authentication at the API layer
  • Trade-offs between separate indices and shared index with filters

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

What's your plan for handling failure modes like empty retrieval results, timeouts mid-request, and stale cached responses?

System DesignRoot Cause Analysis
Author's notes

I actually liked this part of the question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing failure modes as expected events in a production ML system, then systematically address each one with detection, fallback, and recovery strategies. Emphasize graceful degradation, observability, and iterative improvement based on real-world data.

Pro tip: Show that you think about failure modes proactively by designing systems that assume failures will happen, and highlight how you'd use metrics and logs to continuously refine your handling strategies.

1. Acknowledge and Prioritize Failure Modes

List the failure modes mentioned and briefly explain why each is critical in an ML system, showing awareness of their impact on user experience and system reliability.

2. Design Detection and Monitoring

Describe how you would detect each failure mode in real-time, using metrics, logging, and alerting to ensure visibility and quick response.

3. Implement Fallback and Recovery Strategies

For each failure mode, outline specific fallback mechanisms (e.g., default responses, cached results, retries with backoff) and recovery procedures to maintain service continuity.

4. Ensure Graceful Degradation and User Experience

Explain how the system should degrade gracefully, possibly with reduced functionality, while communicating status to users or downstream services.

5. Iterate and Improve Based on Learnings

Emphasize the importance of post-mortems, A/B testing, and continuous monitoring to refine failure handling over time.

Key Points to Mention

  • Empty retrieval results: fallback to a default response or a broader search, and log for analysis.
  • Timeouts mid-request: implement retries with exponential backoff, circuit breakers, and idempotency to avoid duplicate side effects.
  • Stale cached responses: use TTL, cache invalidation strategies, and versioning; consider serving stale data with a warning if fresh data is unavailable.
  • Observability: metrics, logging, tracing, and alerting to detect and diagnose failures quickly.
  • Graceful degradation: design systems to provide partial functionality rather than complete failure.
  • Continuous improvement: use incident reviews and monitoring data to enhance failure handling.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

Define the key metrics you'd track for this system, both offline and online, and design an experiment comparing a BM25 plus cross-encoder re-ranker setup against a dense-only retrieval approach.

A/B Testing & ExperimentationProduct Analytics & Metrics
Author's notes

Metrics felt like home ground.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining offline metrics that measure retrieval quality (e.g., recall@k, nDCG) and online metrics that capture user engagement and business impact (e.g., CTR, dwell time, conversion). Then design an A/B test with careful randomization, sufficient power, and guardrail metrics to compare the two retrieval approaches. Emphasize the trade-offs between offline and online metrics and how they inform iteration.

Pro tip: Highlight the importance of aligning offline metrics with online outcomes and using a pre-registered analysis plan to avoid p-hacking. Mention that at OpenAI, you'd also consider latency and cost as key constraints in production.

1. Define Offline Metrics

Select metrics that evaluate retrieval quality on a labeled dataset, such as recall@k, precision@k, mean reciprocal rank (MRR), and normalized discounted cumulative gain (nDCG). These should reflect the relevance of retrieved documents to the query.

2. Define Online Metrics

Choose metrics that measure user behavior and business impact in production, such as click-through rate (CTR), dwell time, conversion rate, and query success rate. Also include system metrics like latency and cost per query.

3. Design the Experiment

Set up an A/B test with random assignment of users or sessions to control (BM25 + cross-encoder) and treatment (dense-only). Ensure sufficient sample size via power analysis, and define the primary metric and guardrail metrics.

4. Analyze and Interpret

Compare the two approaches on both offline and online metrics. Use statistical tests to determine significance, and check for novelty effects or segment-level differences. Consider trade-offs between relevance, latency, and cost.

5. Iterate and Decide

Based on results, decide whether to adopt the new approach, iterate further, or combine methods. Document learnings and consider long-term impact through holdback experiments.

Key Points to Mention

  • Offline metrics: recall@k, nDCG, MRR, precision@k
  • Online metrics: CTR, dwell time, conversion rate, query success rate, latency, cost
  • A/B testing best practices: randomization unit, power analysis, guardrail metrics
  • Trade-offs between retrieval quality, latency, and computational cost
  • Potential need for hybrid approaches or multi-objective optimization
  • Importance of aligning offline metrics with online business goals

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

Give a back-of-the-envelope capacity estimate for a system handling 10 million documents at 50 QPS. Cover index size, throughput requirements, and rough cost.

System DesignTechnical Trade-offs
Author's notes

10M docs at roughly 512-dimensional float32 embeddings is about 20GB for the raw vectors before any overhead.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying assumptions (document size, query type, latency SLA) and then break the problem into three parts: index size, throughput, and cost. Use simple arithmetic with round numbers to estimate storage, compute, and monthly expenses, and state your reasoning clearly.

Pro tip: Always state your assumptions explicitly and sanity-check the final numbers against known benchmarks (e.g., a single server can handle ~1000 QPS for simple queries). This shows you think like an engineer, not just a calculator.

1. Clarify requirements and assumptions

Ask about document size, query complexity, latency SLA, and whether the system is read-heavy. Assume average document size (e.g., 10 KB) and simple keyword queries for estimation.

2. Estimate index size

Calculate raw data size (10M docs * 10 KB = 100 GB) and multiply by an index overhead factor (e.g., 2-3x for inverted index). Result: ~200-300 GB of index storage.

3. Estimate throughput and servers

Assume each server can handle 500-1000 QPS for simple queries. For 50 QPS, one server suffices, but for redundancy and peak load, use 2-3 servers. Consider sharding if index is large.

4. Estimate cost

Use cloud pricing: storage ~$0.10/GB-month, compute ~$0.10/hour for a mid-tier instance. For 300 GB storage: $30/month; for 3 instances: ~$216/month. Total ~$250/month, plus data transfer and management overhead.

5. Sanity-check and summarize

Verify numbers are reasonable (e.g., 50 QPS is low, so cost is dominated by storage). Summarize key figures and mention potential optimizations like compression or caching.

Key Points to Mention

  • Document size and index overhead factor (e.g., 2-3x raw data)
  • QPS per server and need for redundancy (N+1)
  • Cloud storage and compute pricing (e.g., AWS EC2, S3)
  • Sharding and replication for scalability and availability
  • Caching to reduce load for repeated queries
  • Total cost of ownership including data transfer and management

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q6

If you only had 15 minutes to present a coherent version of this plan, what would you prioritize and what would you explicitly cut?

Roadmap PrioritizationAdaptability & Ambiguity
Author's notes

Honestly a meta-question about prioritization under pressure.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the 15-minute constraint as a forcing function for clarity, then present a ruthlessly prioritized version of the plan that focuses on the highest-impact, highest-uncertainty items. Explicitly state what you're cutting and why, showing that you understand trade-offs and can communicate them to stakeholders.

Pro tip: Tie every prioritization decision back to measurable impact on the end goal—at OpenAI, that often means model performance, safety, or scalability—and be ready to defend your cuts with data or clear reasoning.

1. Clarify the goal and constraints

Briefly restate the plan's objective and the 15-minute constraint to ensure alignment. Emphasize that the goal is a coherent, actionable version, not a comprehensive one.

2. Identify the critical path

Determine the fewest steps that must be taken to achieve a minimally viable outcome. Focus on items that are on the critical path and have the highest impact or risk.

3. Prioritize by impact and uncertainty

Rank remaining items by expected impact and level of uncertainty. Include only those that are essential to validate the core hypothesis or deliver immediate value.

4. Explicitly state what you're cutting

List the items you're omitting and give a concise rationale for each cut, such as lower impact, deferrable, or dependent on unresolved questions.

5. Summarize and invite feedback

Conclude with a crisp summary of the prioritized plan and the cuts, then invite questions or suggestions to adapt further if needed.

Key Points to Mention

  • Focus on the highest-impact, highest-uncertainty items first (e.g., model architecture experiments, data quality checks).
  • Cut or defer lower-priority tasks like extensive hyperparameter tuning, nice-to-have features, or comprehensive documentation.
  • Use a prioritization framework (e.g., impact/effort matrix, MoSCoW) to justify decisions.
  • Communicate trade-offs clearly to stakeholders, emphasizing that cuts are deliberate to ensure coherence and delivery.
  • Be prepared to adapt the plan if new information emerges during the 15-minute presentation.
  • Highlight any quick wins or low-hanging fruit that can be included without compromising the core plan.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.