Bytedance·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Bytedance ML engineer interview that went deep on loss functions, way deeper than I expected for a phone screen. The KL divergence part especially had some follow-ups that made me realize I only half-understood what I thought I knew.

Questions Asked (2)

Walk through the common loss functions (MSE, MAE, Huber, cross-entropy, hinge, focal) and explain when you'd pick each one.

Technical Trade-offsAlgorithms & Data Structures

Author's notes

Started fine with MSE vs MAE, outlier sensitivity, the usual.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer by grouping loss functions by task type (regression vs classification) and then for each, explain the mathematical intuition, robustness properties, and practical trade-offs. Emphasize that the choice depends on the data distribution, presence of outliers, and business metric, and give concrete examples from your experience.

Pro tip: Mention that the loss function should align with the evaluation metric and business objective; for example, if the business cares about large errors, MSE might be better despite outliers, but if robustness is key, Huber or MAE is preferred. Also, note that focal loss is particularly useful for class imbalance in object detection.

1. Categorize by task

Start by grouping loss functions into regression (MSE, MAE, Huber) and classification (cross-entropy, hinge, focal). This sets a clear structure.

2. Explain each loss function

For each, briefly describe the formula, its sensitivity to outliers, gradient behavior, and typical use cases.

3. Discuss trade-offs

Compare them in terms of robustness, convergence speed, and suitability for imbalanced data or specific model types (e.g., SVM with hinge).

4. Provide selection criteria

Give a decision framework: consider data distribution, outliers, class imbalance, and whether the loss is differentiable and compatible with the model.

5. Give real-world examples

Share instances where you chose a particular loss and the outcome, demonstrating practical experience.

Key Points to Mention

MSE: sensitive to outliers, smooth gradient, good for Gaussian noise; MAE: robust to outliers, constant gradient, may converge slowly; Huber: combines MSE and MAE, robust and smooth.
Cross-entropy: standard for classification, works well with probabilistic outputs; hinge: for SVM, maximizes margin, not probabilistic; focal: addresses class imbalance by down-weighting easy examples.
Trade-offs: MSE penalizes large errors more, MAE is robust but less stable near zero, Huber balances both; cross-entropy vs hinge: probabilistic vs margin-based.
Considerations: data distribution (outliers, noise), class imbalance, model type (neural networks vs SVM), and evaluation metric alignment.
Practical tips: use MSE for regression with no outliers, MAE when outliers present, Huber as a compromise; cross-entropy for most classification, hinge for SVMs, focal for detection with extreme imbalance.
Mention that loss function choice affects gradient behavior and thus optimization dynamics, and can be combined with regularization.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Define KL divergence, explain its asymmetry, and describe how it relates to cross-entropy. When would you use it over a simpler loss?

Technical Trade-offsAlgorithms & Data Structures

Author's notes

This is where things got interesting.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start with a precise mathematical definition of KL divergence, then explain its asymmetry with an intuitive example. Connect it to cross-entropy by showing the decomposition, and finally discuss practical scenarios where KL divergence is preferred over simpler losses like MSE.

Pro tip: Emphasize that KL divergence is not a true distance metric due to asymmetry, and mention that in practice, the choice between forward and reverse KL depends on whether you want mode-covering or mode-seeking behavior. This shows deep understanding and practical experience.

1. Define KL Divergence

State the formula: D_KL(P||Q) = Σ P(x) log(P(x)/Q(x)) for discrete distributions, and explain that it measures the inefficiency of assuming Q when the true distribution is P.

2. Explain Asymmetry

Highlight that D_KL(P||Q) ≠ D_KL(Q||P) and provide an intuitive example, such as forward KL penalizing Q for placing low probability where P has mass (zero-avoiding), while reverse KL penalizes Q for placing mass where P has none (zero-forcing).

3. Relate to Cross-Entropy

Show that cross-entropy H(P,Q) = H(P) + D_KL(P||Q). Since H(P) is constant with respect to Q, minimizing cross-entropy is equivalent to minimizing KL divergence when P is fixed.

4. Discuss When to Use KL Over Simpler Losses

Explain that KL divergence is preferred when comparing probability distributions, especially in generative models (VAEs), reinforcement learning (policy optimization), and variational inference, because it accounts for uncertainty and is sensitive to distribution shape, unlike MSE which assumes Gaussian noise and treats all errors equally.

Key Points to Mention

Mathematical definition of KL divergence and its non-negativity (Gibbs' inequality).
Asymmetry and its implications: forward vs reverse KL, mode-covering vs mode-seeking.
Relationship with cross-entropy: H(P,Q) = H(P) + D_KL(P||Q).
KL divergence is not a metric (no triangle inequality, not symmetric).
Use cases: variational inference, generative models (VAEs), policy gradients in RL.
Comparison with MSE: KL is appropriate for probabilities, MSE for continuous values with Gaussian assumptions.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.