Newsbreak·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Jul 2026

Summary

Technical phone screen for an ML Engineer role at NewsBreak. The whole session was basically one deep-dive question on probability calibration, which sounds narrow but they really pushed on the math and the implementation side both.

Questions Asked (1)

Given a trained softmax classifier, derive temperature scaling for calibration: define the temperature-scaled probability p_i(x; T) = softmax(z_i(x) / T), formulate the negative log-likelihood loss on a held-out validation set, derive the gradient with respect to T, and then write Python code to learn T via gradient descent and apply it to calibrate new predictions.

Technical Trade-offsAlgorithms & Data StructuresSystem Design

Author's notes

This was the whole interview, basically.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly defining temperature scaling as a post-hoc calibration method that rescales logits by a single scalar T, then derive the NLL loss and its gradient with respect to T. Finally, implement gradient descent to learn T on a validation set and apply it to calibrate new predictions, emphasizing that T is learned without changing the model's parameters.

Pro tip: Mention that temperature scaling preserves the argmax (accuracy) while improving calibration, and that T is typically optimized on a validation set to avoid overfitting. Also, note that T > 1 softens probabilities, while T < 1 sharpens them.

1. Define temperature scaling and the probability model

Introduce temperature scaling as a post-processing step where logits z_i(x) are divided by a scalar T, and probabilities are computed via softmax(z_i(x)/T). Emphasize that T is a single parameter learned on a validation set.

2. Formulate the negative log-likelihood loss

Write the NLL loss on a held-out validation set: L(T) = -Σ log p_{y_i}(x_i; T), where p_{y_i} is the temperature-scaled probability for the true class y_i.

3. Derive the gradient with respect to T

Compute ∂L/∂T by applying the chain rule: ∂L/∂T = (1/T^2) Σ (z_{y_i} - Σ_j p_j z_j), where p_j are the temperature-scaled probabilities. Show that this gradient is used for optimization.

4. Implement gradient descent to learn T

Write Python code that initializes T=1, computes the loss and gradient on validation logits, and updates T using gradient descent (or a library optimizer) until convergence.

5. Apply calibrated probabilities to new data

Use the learned T to calibrate new predictions by computing softmax(z_i(x)/T). Highlight that this does not change the predicted class, only the confidence estimates.

Key Points to Mention

Temperature scaling is a post-hoc calibration method that does not alter the model's parameters or accuracy.
The NLL loss on validation data is convex in T, ensuring a unique global minimum.
The gradient derivation involves the difference between the true class logit and the expected logit under the scaled distribution.
T is typically optimized using gradient descent or L-BFGS, and T=1 recovers the original softmax probabilities.
Calibration improves the reliability of predicted probabilities, which is crucial for decision-making in applications like news recommendation.
Implementation should use logits (pre-softmax outputs) for numerical stability, and T can be constrained to be positive (e.g., via softplus or parameterizing log T).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.