Pinterest·Machine Learning Engineer·Technical Phone Screen·Intermediate

IntermediatePrefer not to say

Jun 2026

Summary

Pinterest ML engineer interview, just one question worth noting but it was a pretty fundamental one about regularization that I fumbled more than I expected to.

Questions Asked (1)

With L1 and L2 regularization, why does L1 tend to push many model parameters all the way to zero while L2 typically doesn't?

Technical Trade-offsAlgorithms & Data Structures

Author's notes

I knew the answer lived somewhere in the geometry of the penalty terms but I started rambling about sparsity before actually explaining the 'why'.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Explain the geometric and mathematical reasons behind L1 producing sparse solutions while L2 does not. Start with the optimization objective, then contrast the shape of the constraint regions and the behavior of gradients near zero. Conclude with practical implications for feature selection and model interpretability.

Pro tip: Mention that L1's sparsity is not just due to the diamond shape but also because the subgradient at zero includes zero, allowing exact zeros, whereas L2's gradient at zero is zero but the solution is rarely exactly zero due to the smoothness of the penalty. This shows deep understanding beyond the typical geometric explanation.

1. State the optimization problem

Write the regularized loss function: L1 adds λ∑|w_i|, L2 adds λ∑w_i^2. Explain that both aim to minimize loss while keeping weights small.

2. Geometric interpretation

Describe the constraint region: L1 is a diamond (or polytope) with corners on axes, L2 is a circle. The loss contours often touch the diamond at corners, where some weights are exactly zero.

3. Analytical/gradient perspective

For L1, the gradient of the penalty is λ sign(w), which is constant magnitude for non-zero w and undefined at zero (subgradient includes zero). For L2, gradient is 2λw, which goes to zero as w→0. Thus L1 can push weights to exactly zero, while L2 only shrinks them.

4. Sparsity and feature selection

Explain that L1's ability to zero out weights leads to sparse models and automatic feature selection, while L2 results in dense models with small but non-zero weights.

5. Practical implications

Mention when to use each: L1 for interpretability and high-dimensional data, L2 for preventing overfitting when all features are useful, and Elastic Net for combining both.

Key Points to Mention

L1 penalty is the sum of absolute values; L2 is the sum of squares.
Geometric shape: L1 constraint region has sharp corners on axes; L2 is smooth and round.
Gradient/subgradient behavior: L1 has constant gradient magnitude (sign function) and subgradient at zero includes zero; L2 gradient diminishes near zero.
Sparsity: L1 yields exact zeros, leading to feature selection; L2 yields small but non-zero weights.
Bayesian interpretation: L1 corresponds to Laplace prior, L2 to Gaussian prior.
Practical trade-offs: L1 for interpretability, L2 for stability, Elastic Net for combining.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.