Amazon·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Jul 2026

Summary

Amazon ML Engineer screen, one meaty technical question about regularization that sounds deceptively textbook until you're actually in it trying to explain the tradeoffs out loud.

Questions Asked (1)

What is overfitting, why is it a problem, and how do L1 vs L2 regularization address it? Walk through how each modifies the loss function, what effect each has on the model weights, and when you'd choose one over the other.

Technical Trade-offsAlgorithms & Data Structures

Author's notes

I knew this cold on paper but fumbled the 'when would you prefer one over the other' part.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining overfitting and its symptoms, then explain why it harms generalization. Next, describe L1 and L2 regularization mathematically and intuitively, contrasting their effects on weights. Finally, discuss practical selection criteria based on feature sparsity, interpretability, and computational considerations.

Pro tip: Mention that L1 regularization can be solved efficiently using proximal gradient methods, and that in practice, elastic net (a combination of L1 and L2) often outperforms either alone when features are correlated.

1. Define overfitting and its impact

Explain overfitting as when a model learns noise in training data, leading to poor generalization. Mention symptoms like high training accuracy but low validation accuracy.

2. Introduce regularization as a solution

Describe regularization as adding a penalty term to the loss function to constrain model complexity, thus reducing overfitting.

3. Explain L1 regularization (Lasso)

Detail the L1 penalty (sum of absolute weights) added to loss. Explain its effect: drives some weights exactly to zero, producing sparse models and feature selection.

4. Explain L2 regularization (Ridge)

Detail the L2 penalty (sum of squared weights) added to loss. Explain its effect: shrinks weights towards zero but not exactly zero, reducing variance and handling multicollinearity.

5. Compare and choose between L1 and L2

Discuss when to use each: L1 for feature selection and interpretability; L2 for stability and when all features are relevant. Mention elastic net as a hybrid.

Key Points to Mention

Overfitting occurs when model complexity is too high relative to data, capturing noise.
L1 regularization adds λ * sum(|w_i|) to the loss, promoting sparsity.
L2 regularization adds λ * sum(w_i^2) to the loss, promoting small weights.
L1 can zero out weights, effectively performing feature selection.
L2 shrinks weights smoothly, reducing sensitivity to individual features.
Choose L1 for sparse solutions and interpretability; L2 for correlated features and stability.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.