Instacart·Machine Learning Engineer·Technical Phone Screen·Senior

SeniorPrefer not to say

Jun 2026Remote

Summary

ML concepts interview at Instacart for an MLE role. One meaty theory question that turned into a back-and-forth where the interviewer kept pushing until I got to the precise answer. Not as easy as it looks on the surface.

Questions Asked (1)

If you use MSE as your loss function, do vanilla batch gradient descent and SGD converge to the same point?

Technical Trade-offsAlgorithms & Data Structures

Author's notes

I said 'yes' pretty quickly because MSE is convex and I figured both optimizers are just finding the same minimum.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Clarify that with MSE loss, the optimization problem is convex, so both batch gradient descent and SGD converge to the global minimum given appropriate learning rates and sufficient iterations. However, they may take different paths and have different convergence rates, and in practice, SGD may oscillate around the minimum due to noise. Emphasize that the final point should be the same in expectation, but stochasticity can lead to small differences unless learning rate is decayed.

Pro tip: Mention that while the theoretical global minimum is the same, SGD's noise can cause it to settle at a slightly different point unless you use learning rate decay or averaging. This shows you understand practical nuances beyond textbook theory.

1. Identify the optimization problem

State that with MSE loss, the objective is convex (for linear models) or non-convex (for neural networks), but assume convexity for this discussion. For convex problems, there is a unique global minimum.

2. Explain batch gradient descent behavior

Batch gradient descent computes the exact gradient over the entire dataset and moves deterministically toward the global minimum, converging to it given a suitable learning rate and enough iterations.

3. Explain SGD behavior

SGD uses noisy gradients from mini-batches, so it follows a stochastic path. With a fixed learning rate, it oscillates around the minimum; with a decaying learning rate, it converges to the global minimum in expectation.

4. Compare convergence points

In theory, both converge to the same global minimum for convex MSE. In practice, SGD may end up very close but not exactly at the minimum due to noise, unless learning rate is annealed or averaging is used.

5. Address non-convexity caveat

If the model is non-convex (e.g., neural networks), MSE loss may have multiple local minima, and batch GD and SGD could converge to different local minima depending on initialization and batch order.

Key Points to Mention

Convexity of MSE loss for linear models ensures a unique global minimum.
Batch gradient descent is deterministic and converges to the global minimum given appropriate learning rate.
SGD introduces noise, causing oscillations around the minimum; convergence to the exact minimum requires learning rate decay.
In expectation, SGD converges to the same point as batch GD for convex problems.
For non-convex problems (e.g., deep networks), different minima may be reached.
Practical considerations: learning rate schedules, batch size, and averaging can affect final point.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.