Openai·Software Engineer·Onsite - System Design / Architecture·Senior

Senior

May 2026

Summary

System design round at OpenAI for a software engineering role. The whole thing centered on designing something like Sora, with a lot of back-and-forth on scheduling, what happens when things break, and how you'd monitor the system in production.

Questions Asked (1)

Design a large-scale video generation system similar to Sora, covering job scheduling, failure handling, and observability.

System DesignTechnical Trade-offsRoot Cause Analysis

Author's notes

This is a meaty one and I don't think I fully appreciated how deep the failure scenarios would go.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale, then propose a high-level architecture with a distributed job scheduler, fault-tolerant workers, and observability stack. Dive into trade-offs for scheduling (e.g., priority vs. fairness), failure handling (retries, checkpointing), and observability (metrics, tracing, logging).

Pro tip: Emphasize that video generation is long-running and resource-intensive, so design for preemption and checkpointing to avoid wasted compute. Also, discuss how to handle partial failures gracefully to maintain user experience.

1. Clarify Requirements and Scale

Ask questions to understand expected throughput, latency, video length, resolution, and budget constraints. Establish assumptions about cluster size and model complexity.

2. High-Level Architecture

Outline components: API gateway, job queue, scheduler, worker pool (GPU/TPU), storage for checkpoints and outputs, and observability pipeline. Explain how they interact.

3. Job Scheduling Design

Discuss scheduling policies (priority, fairness, preemption), resource allocation (GPU memory, CPU), and queue management. Consider using a distributed scheduler like Kubernetes or custom.

4. Failure Handling and Recovery

Describe strategies for worker failures (retries, checkpointing, idempotency), job failures (dead-letter queues, alerting), and data consistency. Mention graceful degradation.

5. Observability and Monitoring

Cover metrics (latency, throughput, error rates), logging (structured logs), tracing (distributed tracing), and alerting. Explain how to debug issues and ensure SLA compliance.

Key Points to Mention

Use of a distributed job queue (e.g., Kafka, RabbitMQ) and scheduler (e.g., Kubernetes, YARN) for scalability.
Checkpointing and preemption to handle long-running jobs and spot instance interruptions.
Idempotent job execution and exactly-once semantics for reliability.
Observability stack: Prometheus for metrics, Grafana for dashboards, Jaeger for tracing, ELK for logging.
Trade-offs between latency and throughput, cost vs. performance, and fairness vs. priority.
Autoscaling of workers based on queue depth and resource utilization.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.