Upstart·Machine Learning Engineer·Technical Phone Screen·Senior

SeniorPrefer not to say

Apr 2026

Summary

Interviewed for an ML Engineer role at Upstart and got hit with a pretty deep question on logistic regression loss functions. Not a casual screen by any means.

Questions Asked (1)

What is the loss function used in logistic regression? Walk through the binary cross-entropy expression, derive it from a probabilistic perspective using maximum likelihood, and explain why it's used instead of mean squared error for classification tasks. Bonus if you can extend the idea to the multi-class case.

Technical Trade-offsAlgorithms & Data Structures

Author's notes

This one went deeper than I expected for a phone screen.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by stating that logistic regression uses binary cross-entropy (log loss), then derive it from maximum likelihood estimation assuming a Bernoulli distribution. Explain why MSE is unsuitable for classification due to non-convexity and poor gradient behavior, and optionally extend to multi-class with categorical cross-entropy.

Pro tip: Emphasize that cross-entropy is convex for logistic regression, ensuring reliable optimization, and mention that it penalizes confident wrong predictions heavily, which is crucial for classification.

1. State the loss function

Clearly identify binary cross-entropy as the loss function used in logistic regression.

2. Write the expression

Present the binary cross-entropy formula: L(y, p) = -[y log(p) + (1-y) log(1-p)].

3. Derive from maximum likelihood

Assume labels are Bernoulli distributed with probability p = sigmoid(w·x + b). Write the likelihood, take the negative log, and show it equals the cross-entropy loss.

4. Compare with MSE

Explain that MSE leads to a non-convex loss surface for logistic regression, causing optimization issues, and that cross-entropy provides stronger gradients for misclassified points.

5. Extend to multi-class

Generalize to categorical cross-entropy: L = -Σ y_i log(p_i), where p_i = softmax output, derived from multinomial likelihood.

Key Points to Mention

Binary cross-entropy formula and its probabilistic interpretation
Maximum likelihood estimation derivation: likelihood, log-likelihood, negative log-likelihood
Sigmoid function as the probability model
Why MSE is non-convex for logistic regression and leads to slow convergence
Cross-entropy's convexity and better gradient properties
Multi-class extension: categorical cross-entropy and softmax

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.