I went in expecting a leetcode-style session and got hit with a full system design question instead.
Start by clarifying requirements and constraints, then walk through the end-to-end RAG pipeline: document ingestion, embedding, indexing, retrieval, prompt construction, and evaluation. Emphasize trade-offs at each stage and how you would iterate based on metrics.
Pro tip: Anchor your design to a concrete use case (e.g., Geico policy Q&A) and discuss how you'd handle domain-specific challenges like long documents, jargon, and compliance. Show that you measure success with both retrieval and generation metrics.
Ask about document types, volume, update frequency, latency, and accuracy needs. This shapes choices like embedding model, index type, and whether to use a vector DB or a simpler solution.
Describe chunking strategies (e.g., fixed-size with overlap, semantic) and embedding model selection (e.g., OpenAI, Sentence Transformers). Mention handling of metadata and preprocessing.
Explain indexing options (e.g., FAISS, Pinecone, Elasticsearch) and retrieval methods (dense, sparse, hybrid). Discuss top-k, re-ranking, and filtering by metadata.
Detail how to format retrieved context into a prompt for the LLM, including instructions to reduce hallucination. Mention techniques like few-shot examples and chain-of-thought.
Outline metrics for retrieval (recall@k, MRR) and generation (faithfulness, relevance). Describe offline and online evaluation, and how to use feedback to improve the pipeline.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.