Anthropic·Software Engineer·Onsite - System Design / Architecture
Jun 2026
System design round at Anthropic for a software engineer role, focused entirely on building an inference API service for a large model. The interviewer pushed hard on implementation specifics, not just high-level architecture, which caught me a bit off guard.
- Design an inference API service that serves a large model (like an LLM or vision model) behind a request/response interface. Walk through API design, request batching, scheduling, autoscaling, model loading, GPU vs CPU trade-offs, monitoring, and SLOs.
“I started with the API surface and worked outward, which felt natural, but I underestimated how deep they wanted to go on the GPU execution side.” The rest of the author's notes on Software Engineer interview at Anthropic, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post