Anthropic·Software Engineer·Onsite - System Design / Architecture
Apr 2026
System design round at Anthropic for a software engineer role, focused entirely on batch inference for large ML models. Pretty deep dive, they clearly wanted to see if you'd thought about GPU efficiency and failure recovery at scale, not just the happy path.
- Design a batch inference system for a large language model that processes millions of offline requests as efficiently as possible, including job submission, status tracking, output persistence, and retry logic for failed items.
- How would you maximize GPU utilization across a large batch workload with variable-length inputs, including your approach to batching strategy and memory management?
- How would you handle multi-tenant scheduling with priority levels, quotas, and potential preemption across competing batch jobs?
- What observability would you build into this system, and what metrics matter most for understanding whether the system is healthy and cost-efficient?
“This is a beast of a question.” The rest of the author's notes on Software Engineer interview at Anthropic, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post