Google·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

Google ML engineer interview focused entirely on system design for a hybrid-source chatbot. One long question, lots of sub-parts, and I spent way too much time on retrieval before realizing they wanted me to talk about evaluation and feedback loops too.

Questions Asked (1)

Design an ML system for a chatbot that can answer user questions using both structured data sources (like databases and tabular data) and unstructured sources (like documents and web pages). Walk through query understanding and routing, retrieval strategies including vector search, keyword search, and text-to-SQL, RAG-based grounding, LLM selection, evaluation metrics like faithfulness and answer correctness, latency considerations, and how you'd build in feedback loops.

System DesignTechnical Trade-offsA/B Testing & Experimentation

Author's notes

This was one question but it had like eight sub-questions hiding inside it.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints (data sources, latency, accuracy, scale), then walk through the end-to-end pipeline: query understanding, routing, retrieval (vector, keyword, text-to-SQL), RAG grounding, LLM selection, and evaluation. Emphasize trade-offs at each stage and how you'd iterate with feedback loops and A/B testing.

Pro tip: Anchor your design in concrete metrics (e.g., faithfulness, answer correctness, latency) and explicitly discuss how you'd handle failure modes like hallucination or retrieval misses. Show that you think about the system holistically, not just the model.

1. Clarify Requirements and Scope

Ask about data sources, query types, latency SLAs, scale, and accuracy targets. Define success metrics and constraints to guide design decisions.

2. Design Query Understanding and Routing

Explain how to parse and classify queries (intent, entity extraction) and route them to appropriate retrieval methods (structured vs. unstructured).

3. Implement Retrieval Strategies

Detail vector search for semantic retrieval, keyword search for exact matches, and text-to-SQL for structured data. Discuss hybrid approaches and re-ranking.

4. Integrate RAG and Select LLM

Describe how to ground LLM responses with retrieved evidence, choose an LLM based on cost, latency, and quality, and handle context window limits.

5. Evaluate and Iterate with Feedback Loops

Define offline and online metrics (faithfulness, answer correctness, latency), set up A/B testing, and incorporate user feedback for continuous improvement.

Key Points to Mention

Query understanding: intent classification, entity recognition, and routing logic
Retrieval: hybrid search (vector + keyword), text-to-SQL for structured data, re-ranking
RAG grounding: citation, context selection, and hallucination mitigation
LLM selection: trade-offs between model size, latency, cost, and accuracy
Evaluation metrics: faithfulness, answer correctness, latency, and user satisfaction
Feedback loops: A/B testing, user feedback, and continuous model improvement

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.