Anthropic·Software Engineer·Onsite - System Design / Architecture
May 2026
System design round at Anthropic for a software engineer role, centered entirely on designing a prompt playground product from scratch. The question was deceptively broad and pushed hard on product thinking, streaming architecture, and scale, all at once.
- Design a prompt playground web product where developers can write prompts, tune model parameters, run generations that stream back token by token, save and version their prompts, and copy the resulting API call.
- How would you design the backend request path so that tokens stream from the model back to the browser in real time? What transport mechanism would you use and why?
- How do you scale this system to millions of concurrent users, given that each generation is a long-lived streaming connection and GPU inference capacity is the real bottleneck?
- How would you implement side-by-side comparison of two prompt variations, streaming both concurrently and showing their outputs and costs aligned?
- Where can caching actually help in a generative AI product, and where does it break down?
- How do you enforce fair per-user quotas and stop a single user from monopolizing inference capacity by running many long generations concurrently?
- A generation streams 800 tokens and the connection drops at token 600. What does the client experience, and how do you let it resume without re-running the whole generation?
“This one is bigger than it looks.” The rest of the author's notes on Software Engineer interview at Anthropic, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post