Harvey·Software Engineer·Onsite - System Design / Architecture
Apr 2026
System design interview at Harvey for a software engineer role, focused entirely on building a RAG-based Q&A agent for a big law firm's memo corpus. It was a long open-ended design discussion with multiple layers, and the interviewer clearly wanted breadth across ingestion, retrieval, grounding, and evaluation rather than a deep dive into any single piece.
- Design an AI system that lets attorneys ask natural-language questions and get answers grounded in a large corpus of legal memos, covering ingestion, retrieval, answer generation, and evaluation.
- Your crawled corpus has two memos from different firms that directly contradict each other on the same legal question. How should the agent respond, and how do you surface that conflict to the user?
- A regulator publishes a change that supersedes the conclusions in several memos already in your index. How does your pipeline prevent the agent from citing those stale memos as current authority?
- Attorneys say the agent refuses too often on questions it could partially answer. How do you tune the refusal threshold without increasing hallucinations, and how do you measure that trade-off?
- How would your design change if query volume grew 100x, or if you needed to support multi-turn conversations where follow-up questions depend on prior context?
“This was the whole interview basically.” The rest of the author's notes on Software Engineer interview at Harvey, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post