Harvey·Software Engineer·Onsite - System Design / Architecture
Jul 2026
System design interview at Harvey for a software engineer role, centered entirely on building a RAG pipeline for legal research. The problem was meaty and touched basically every layer of the stack, from crawling to eval. Felt like a real product they're thinking about.
- Design a retrieval-augmented generation system that answers legal questions by grounding responses in publicly published memos and client alerts from large law firms, with citations back to source documents.
- How do you design the crawling and ingestion pipeline to keep the corpus fresh across roughly 500 heterogeneous law firm websites without re-crawling everything constantly?
- Walk through your chunking strategy and how you'd structure the retrieval index to maximize both recall and precision for legal queries.
- How do you constrain the LLM to only answer from retrieved evidence, and how do you verify that citations in the generated answer actually support the claims being made?
- How do you evaluate this system offline and online, and how do you isolate whether a bad answer is a retrieval failure or a generation failure?
- A memo gets corrected or retracted after you've already indexed it. How do you detect this and make sure users aren't getting answers backed by stale or withdrawn guidance?
- Your retrieval recall looks good in offline eval but users are still reporting wrong answers. How do you figure out where the failure is happening?
- How would you extend this system to support comparative queries, like asking how different firms differ in their guidance on a specific topic?
“This is the main question and it's a beast.”