Start by clarifying the use case and constraints (e.g., document types, query volume, latency needs) since HarveyAI operates in the legal domain where accuracy and citations are critical. Then outline a modular RAG pipeline: ingestion, indexing, retrieval, and generation, explaining each component and the trade-offs you'd make in a notebook environment. Emphasize evaluation and iteration, showing how you'd measure and improve retrieval and generation quality.
Pro tip: In a notebook, use small, representative samples and pre-computed embeddings to iterate quickly, but explicitly call out how you'd scale each component (e.g., vector DB, batch processing) in production. Also, mention that for legal documents, you'd implement citation-aware generation to ensure every claim is traceable to a source.
Ask about document types, query patterns, latency, and accuracy requirements. For HarveyAI, highlight the need for high precision, citation, and handling of long legal texts.
Explain how you'd load and preprocess documents (e.g., chunking with overlap, metadata extraction), then generate embeddings and store them in a vector index. Discuss chunking strategies and embedding model choices.
Describe the retrieval step: embed the query, search the vector index, and optionally re-rank results with a cross-encoder. Mention hybrid search (keyword + vector) for legal terms.
Show how to construct a prompt with retrieved chunks and generate an answer using an LLM. Emphasize citation and handling of context window limits.
Outline an evaluation plan: retrieval metrics (recall@k, MRR) and generation metrics (faithfulness, answer relevance). Use a small labeled set and iterate on chunking, models, and prompts.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.