System design round at OpenAI for a software engineering role. The whole interview was basically one giant question about LLM inference infrastructure, and it went deep fast. Not a casual conversation.
“I started with the API gateway and worked toward the GPU pool, covering auth, quota enforcement, and a request queue in between.” The rest of the author's notes on Software Engineer interview at OpenAI, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.