← HarveyAI Interview Insights

HarveyAI·Software Engineer·Onsite - System Design / Architecture·Senior

Senior
May 2026

Summary

System design round at HarveyAI for a software engineer role, focused entirely on building a RAG pipeline from ingestion to answer generation. Dense technical territory and they clearly wanted you to go deep on every layer, not just wave at the problem.

Questions Asked (5)

Q1

Design a retrieval-augmented generation system that grounds an LLM's responses in an external knowledge base, covering ingestion, retrieval, and answer generation end to end.

System DesignTechnical Trade-offs
Author's notes

This is a beast of a question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements (e.g., domain, latency, scale, accuracy) and then walk through the end-to-end pipeline: ingestion, indexing, retrieval, and generation. Emphasize trade-offs at each stage, such as chunking strategies, embedding models, and reranking, and how they impact answer quality and system performance.

Pro tip: Always ground your design in concrete metrics (e.g., recall@k, latency, cost) and discuss how you would evaluate and iterate on the system, showing a data-driven mindset.

1. Clarify Requirements and Constraints

Ask about the knowledge base size, update frequency, query types, latency and cost constraints, and accuracy expectations to tailor the design.

2. Design Ingestion Pipeline

Outline document parsing, chunking, embedding generation, and indexing. Discuss trade-offs in chunk size, overlap, and metadata enrichment.

3. Design Retrieval System

Choose between sparse (e.g., BM25) and dense (e.g., embeddings) retrieval, or hybrid. Explain indexing (e.g., FAISS, HNSW), query processing, and reranking strategies.

4. Design Answer Generation

Describe how retrieved passages are fed into the LLM, prompt engineering for grounding, and techniques to reduce hallucination (e.g., citation, self-check).

5. Address Evaluation, Monitoring, and Iteration

Propose metrics (e.g., retrieval recall, answer faithfulness) and a feedback loop for continuous improvement, including A/B testing and user feedback.

Key Points to Mention

  • Chunking strategies and their impact on retrieval quality
  • Embedding model selection and fine-tuning for domain-specific data
  • Hybrid retrieval combining sparse and dense methods
  • Reranking and query expansion techniques
  • Prompt design to ensure grounding and citation
  • Evaluation metrics and monitoring for RAG systems

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

How would you handle document ingestion from multiple source types like PDFs, web pages, and internal wikis, and what chunking strategy would you use?

System DesignTechnical Trade-offs
Author's notes

Talked through fixed-size chunks with overlap versus semantic boundary splitting.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by outlining a modular ingestion pipeline with source-specific adapters that normalize content into a unified format, then discuss chunking as a trade-off between retrieval granularity and context preservation. Emphasize that chunking should be content-aware and evaluated empirically for the target use case.

Pro tip: Mention that chunking strategy should be driven by retrieval evaluation metrics (e.g., recall@k, answer faithfulness) rather than intuition, and that you'd A/B test strategies like semantic vs. fixed-size chunking. Also note the importance of preserving document structure (headings, tables) during ingestion to improve chunk quality.

1. Identify source types and extraction challenges

Discuss the unique characteristics of PDFs (scanned vs. digital, tables, multi-column), web pages (boilerplate, dynamic content), and internal wikis (structured markup, permissions). Highlight the need for specialized parsers and fallback OCR.

2. Design a modular ingestion pipeline

Propose a pipeline with pluggable source adapters that extract raw text and metadata, followed by normalization (e.g., cleaning, deduplication) and enrichment (e.g., adding source, timestamp). Emphasize scalability and fault tolerance.

3. Choose a chunking strategy based on content and use case

Explain that chunking should balance semantic coherence and size limits. For technical documents, use structure-aware chunking (e.g., by sections); for general text, consider recursive character splitting or semantic chunking with embeddings.

4. Address trade-offs and evaluation

Discuss trade-offs: smaller chunks improve retrieval precision but may lose context; larger chunks preserve context but reduce precision. Propose evaluating with retrieval metrics and end-to-end QA performance.

5. Implement monitoring and iteration

Suggest logging chunk statistics (size distribution, overlap) and monitoring retrieval quality. Plan for iterative improvements based on user feedback and performance data.

Key Points to Mention

  • Use of OCR for scanned PDFs and handling of tables/figures
  • Web scraping considerations: respecting robots.txt, handling JavaScript-rendered content
  • Wiki ingestion: leveraging APIs, handling permissions and access control
  • Chunking methods: fixed-size, recursive, semantic, and structure-aware
  • Overlap between chunks to maintain context
  • Metadata preservation (source, section headings) for filtering and citation
  • Evaluation metrics: retrieval recall, precision, and downstream QA accuracy

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

Walk me through how you'd combine dense vector search with sparse retrieval like BM25, and how reranking fits in.

System DesignTechnical Trade-offsAPI & Integrations
Author's notes

This one I actually felt decent about.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem: dense vectors capture semantic similarity while BM25 excels at exact keyword matching, so combining them improves recall. Then describe a concrete pipeline: parallel retrieval, fusion (e.g., reciprocal rank fusion), and a cross-encoder reranker for precision. Finally, discuss trade-offs like latency, cost, and tuning weights.

Pro tip: Emphasize that reranking is not just a final step but a critical precision booster; mention that you'd evaluate the system end-to-end with metrics like nDCG and MRR, and be ready to discuss how you'd handle latency budgets by parallelizing retrieval and using a lightweight reranker.

1. Clarify requirements and constraints

Ask about the use case (e.g., legal document search), latency requirements, and data characteristics to tailor the hybrid approach.

2. Describe dense and sparse retrieval

Explain how dense vectors (e.g., from bi-encoders like SBERT) capture semantics, while BM25 captures lexical matches; note their complementary strengths.

3. Outline hybrid retrieval pipeline

Detail parallel retrieval from both methods, then fusion using techniques like reciprocal rank fusion or weighted sum of scores.

4. Explain reranking

Describe using a cross-encoder (e.g., BERT-based) to rerank top-k results from fusion, improving precision at the cost of latency.

5. Discuss trade-offs and evaluation

Cover latency vs. accuracy, cost of reranking, tuning fusion weights, and evaluation metrics like nDCG, MRR, and recall@k.

Key Points to Mention

  • Dense retrieval (bi-encoders) vs. sparse retrieval (BM25) and their complementary nature
  • Fusion techniques: reciprocal rank fusion, weighted score combination, or learned fusion
  • Reranking with cross-encoders for precision, and the latency-accuracy trade-off
  • Handling latency: parallel retrieval, caching, and using lightweight rerankers or distillation
  • Evaluation metrics: nDCG, MRR, recall@k, and A/B testing
  • Scalability considerations: indexing, sharding, and approximate nearest neighbor (ANN) for dense vectors

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

How would you evaluate the quality of a RAG system, both at the retrieval layer and the final answer quality?

Product Analytics & MetricsA/B Testing & Experimentation
Author's notes

Went with recall@k and MRR for retrieval, then faithfulness and answer relevance for end-to-end.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by breaking down the RAG pipeline into retrieval and generation components, then define separate metrics for each. For retrieval, focus on ranking and relevance metrics; for generation, assess faithfulness, relevance, and correctness. Emphasize the importance of end-to-end evaluation and continuous monitoring in production.

Pro tip: In practice, retrieval quality often dominates final answer quality, so invest in robust retrieval evaluation first. Also, consider using LLM-based evaluation for generation metrics, but validate with human judgments periodically.

1. Decompose the RAG pipeline

Clearly separate the retrieval and generation stages to evaluate each independently. This helps isolate issues and attribute performance.

2. Define retrieval metrics

Use metrics like recall@k, precision@k, MRR, and NDCG to measure how well the retriever fetches relevant documents. Consider domain-specific relevance judgments.

3. Define generation metrics

Evaluate answer quality using faithfulness (does the answer stick to retrieved evidence?), answer relevance, and correctness. Use both automatic metrics (e.g., BLEU, ROUGE, BERTScore) and LLM-based or human evaluations.

4. Evaluate end-to-end performance

Measure the final answer quality in the context of the full system, e.g., through human evaluation or task-specific success metrics. This captures interactions between retrieval and generation.

5. Implement continuous monitoring and A/B testing

Set up dashboards to track metrics over time and run A/B tests to compare system changes. Use online metrics like user engagement or task completion rates.

Key Points to Mention

  • Retrieval metrics: recall@k, precision@k, MRR, NDCG
  • Generation metrics: faithfulness, answer relevance, correctness
  • Use of LLM-based evaluation (e.g., GPT-4 as judge) with human validation
  • End-to-end evaluation and task-specific success metrics
  • A/B testing and online monitoring for continuous improvement
  • Handling of domain-specific challenges (e.g., legal documents at HarveyAI)

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

What operational concerns would you flag for a RAG system handling sensitive enterprise data, particularly around access control and PII?

System DesignAdaptability & Ambiguity
Author's notes

Shorter exchange.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the core tension: RAG systems improve answer quality by retrieving from a broad corpus, but that same retrieval can leak data across permission boundaries. Then walk through the operational lifecycle—ingestion, indexing, retrieval, generation, and monitoring—highlighting access control and PII risks at each stage, and close with concrete mitigations like permission-aware retrieval and PII redaction.

Pro tip: Emphasize that access control must be enforced at query time, not just at ingestion, because user permissions change and documents get reclassified. Mention that you'd log retrieval decisions (which chunks were fetched and why) to enable audits and incident response.

1. Map the data lifecycle and trust boundaries

Identify where sensitive data enters, is stored, and is retrieved. Clarify who can access what and where permission checks must occur.

2. Enforce permission-aware retrieval

Ensure the retriever filters documents based on the user's current entitlements, not just document-level ACLs. Consider row-level security and query-time policy evaluation.

3. Detect and redact PII

Implement PII detection at ingestion and query time, with redaction or tokenization before embedding and generation. Handle false positives and negatives gracefully.

4. Secure the vector store and embeddings

Treat embeddings as sensitive data: encrypt at rest, isolate tenants, and prevent embedding inversion attacks. Control access to the vector database itself.

5. Monitor, audit, and test

Log retrieval and generation events for audit trails. Run regular red-team exercises to test for data leakage and permission bypasses.

Key Points to Mention

  • Permission-aware retrieval: filtering at query time based on user identity and current ACLs.
  • PII detection and redaction: using tools like Presidio or custom NER, and handling redaction before embedding.
  • Embedding security: encrypting vector stores, preventing inversion attacks, and tenant isolation.
  • Audit logging: recording which documents were retrieved for which queries to enable forensics.
  • Data leakage via generation: ensuring the LLM doesn't regurgitate sensitive data from context.
  • Compliance and governance: aligning with GDPR, HIPAA, or internal policies, and handling data subject requests.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.