OpenAI·Software Engineer·Onsite - System Design / Architecture
May 2026
System design round at OpenAI for a software engineering role, focused entirely on designing a GPU job scheduler for a text-to-video generation service. Pretty deep dive, multiple parts, and the follow-ups got increasingly brutal toward the end.
- Walk through the full request lifecycle for a long-running GPU generation job: how does a submission get accepted, where does it wait, how does it get dispatched to a worker, and how does the client get status and results back?
- How do you design the scheduling policy across multiple user tiers with different SLOs, while making sure no tier gets completely starved under sustained load?
- How do you enforce per-tier and per-customer quotas at admission so the scheduler isn't overwhelmed trying to be fair across traffic it should have rejected upstream?
- When a high-priority job arrives and all GPUs are occupied, how do you decide which running job to preempt, and do you checkpoint it or just kill and restart?
- How do you allocate GPUs to multi-GPU jobs without causing fragmentation or leaving GPUs idle while a job waits for the rest of its allocation?
- One enterprise customer suddenly submits 10,000 jobs at once. How do you protect other enterprise customers and lower tiers, and what does the offending customer actually experience?
- If you wanted to add GPU fleet autoscaling, how does the multi-minute provisioning lag change your queuing and preemption strategy?
- Your fleet has multiple GPU SKUs and some jobs only run on certain hardware. How does the scheduler handle job-to-hardware compatibility?
- Identical prompts and parameters come in repeatedly. How would you add result caching or deduplication, and what are the correctness and privacy risks?
- How do you keep the scheduler itself highly available, and what prevents a job from being dispatched to two workers simultaneously after a scheduler crash?
“This is the foundation question and I spent probably too long here.” The rest of the author's notes on Software Engineer interview at OpenAI, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post