Anthropic·Software Engineer·Onsite - System Design / Architecture
- Design a multithreaded web crawler that starts from a seed URL, crawls all reachable pages under the same hostname without revisiting any URL, and uses a worker pool pulling from a shared queue. Cover thread-safe deduplication, queue design, graceful shutdown, rate limiting, and when you'd choose threads vs async vs processes.
“This is a meaty one and it looks straightforward until you actually have to talk through all the moving parts at once.”