Zillow·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Apr 2026

Summary

Zillow ML engineer interview, technical phone screen focused on model behavior fundamentals. One meaty question about hallucinations that covered a lot of ground. Felt like they wanted to see if you actually understood the mechanics or just knew the buzzword.

Questions Asked (1)

LLMs are known to hallucinate. Walk through why this happens, covering training objectives, model-level limitations, inference-time factors, and what techniques can reduce it in practice.

Technical Trade-offsRoot Cause AnalysisSystem Design

Author's notes

This question is broader than it sounds and I kind of learned that the hard way.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer around the four areas the question specifies: training objectives, model-level limitations, inference-time factors, and mitigation techniques. For each, explain the root cause and connect it to practical implications, using concrete examples where possible. Conclude by discussing trade-offs of different mitigation strategies and how you would choose among them in a production system.

Pro tip: Emphasize that hallucination is not a single bug but an emergent property of probabilistic training and generation; show maturity by discussing how to measure and monitor hallucination rates in production, and how to set acceptable thresholds based on business risk.

1. Training Objectives

Explain that LLMs are trained to maximize likelihood of the next token, not to be truthful. This objective rewards plausible continuations, even if factually incorrect, because the model learns statistical patterns rather than a grounded world model.

2. Model-Level Limitations

Discuss how the model's knowledge is frozen at training time, lacks real-time verification, and may have gaps or biases in its training data. Also mention that the softmax output is a probability distribution over tokens, not a confidence measure of factual accuracy.

3. Inference-Time Factors

Cover how decoding strategies (e.g., temperature, top-k, top-p) affect hallucination: higher randomness increases creativity but also fabrication. Also note that the model has no inherent mechanism to say 'I don't know' and may overgeneralize from limited context.

4. Mitigation Techniques

List practical techniques: retrieval-augmented generation (RAG) to ground responses, fine-tuning on domain-specific data, prompt engineering (e.g., chain-of-thought, self-consistency), and post-hoc verification (e.g., fact-checking APIs, ensemble methods).

5. Trade-offs and Production Considerations

Discuss trade-offs: RAG adds latency and complexity, fine-tuning requires labeled data, and verification may not scale. Emphasize the need to measure hallucination rates, set thresholds, and choose techniques based on use case and risk tolerance.

Key Points to Mention

Maximum likelihood estimation (MLE) training rewards plausibility over truthfulness.
Lack of grounding in real-world facts and inability to verify statements during generation.
Decoding parameters (temperature, top-p) control randomness and directly impact hallucination rate.
Retrieval-augmented generation (RAG) as a primary mitigation by providing external context.
Fine-tuning and reinforcement learning from human feedback (RLHF) can reduce but not eliminate hallucinations.
Evaluation metrics like factual consistency, hallucination rate, and human evaluation are essential for monitoring.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.