Snapchat·Machine Learning Engineer·Technical Phone Screen·Senior

SeniorPrefer not to say

Apr 2026

Summary

Snapchat ML engineer screen, one meaty normalization question that ended up being more involved than I expected. Felt okay about it but definitely left some things on the table.

Questions Asked (1)

What are the differences between Layer Normalization and Batch Normalization? Walk through how each one works, when you'd use one over the other, and how they behave differently at training versus inference.

Technical Trade-offsSystem Design

Author's notes

Started okay, explained that BN normalizes across the batch dimension while LN normalizes across the feature dimension for each sample independently.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining both normalization techniques and their core mechanics, then contrast their behavior during training and inference, and finally discuss practical trade-offs and use cases. Use a structured comparison to highlight when each is preferred, tying back to real-world scenarios like Snapchat's recommendation or vision models.

Pro tip: Mention that Batch Norm's dependence on batch statistics makes it sensitive to batch size and non-i.i.d. data, while Layer Norm's per-sample normalization is crucial for sequence models and small batches. Also note that in inference, Batch Norm uses running averages, whereas Layer Norm computes statistics on-the-fly.

1. Define and Explain Mechanics

Clearly define Batch Normalization (normalizes across batch dimension per feature) and Layer Normalization (normalizes across feature dimension per sample). Explain how each computes mean and variance.

2. Training vs. Inference Behavior

Describe how Batch Norm uses batch statistics during training but running averages during inference, while Layer Norm uses per-sample statistics consistently in both phases.

3. Use Cases and Trade-offs

Discuss when to use each: Batch Norm for CNNs with large batches, Layer Norm for RNNs/Transformers and small batches. Highlight trade-offs like batch size sensitivity, memory, and parallelization.

4. Connect to Real-World Systems

Relate to practical scenarios, e.g., Snapchat's use of Transformers for text or vision models, emphasizing why Layer Norm is common in NLP and Batch Norm in vision.

Key Points to Mention

Batch Norm normalizes across the batch dimension for each feature; Layer Norm normalizes across features for each sample.
Batch Norm uses batch statistics during training and running averages during inference; Layer Norm uses per-sample statistics in both.
Batch Norm is sensitive to batch size and works well for CNNs; Layer Norm is batch-independent and suited for RNNs/Transformers.
Layer Norm is preferred in sequence models due to variable sequence lengths and small batches.
Batch Norm can improve convergence and allow higher learning rates; Layer Norm stabilizes hidden state dynamics in recurrent models.
In inference, Batch Norm's running averages can cause train-test discrepancy if not properly estimated; Layer Norm has no such issue.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.