Amazon·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

May 2026

Summary

Amazon ML engineer interview that was pretty much a deep dive into linear regression from scratch. One question, but they really wanted you to go all the way from loss definition down to SGD pitfalls, so it took a while.

Questions Asked (1)

Given a dataset of 1D samples, fit a linear model by minimizing MSE. Walk through defining the loss, deriving gradients for both parameters via backprop-style reasoning, implementing SGD training with initialization and update rules, and discuss pitfalls like scaling and divergence.

Algorithms & Data StructuresTechnical Trade-offsSystem Design

Author's notes

This looks like a textbook question until you realize they want the full chain: write out the MSE, differentiate with respect to both slope and intercept, then actually talk through SGD like you'd implement it, not just recite the update rule.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by formally defining the linear model and MSE loss, then derive gradients for both parameters using the chain rule, mirroring backpropagation. Next, outline the SGD algorithm with initialization and update rules, and finally discuss practical pitfalls like feature scaling and learning rate selection, including how to detect and address divergence.

Pro tip: Emphasize that feature scaling is crucial for SGD convergence; without it, the loss surface becomes elongated, causing slow convergence or divergence. Mention that monitoring the loss curve and using techniques like learning rate decay or adaptive optimizers can mitigate these issues.

1. Define the model and loss

State the linear model: y_pred = w * x + b. Define the MSE loss: L = (1/N) * sum((y_pred - y_true)^2).

2. Derive gradients

Compute gradients of L w.r.t. w and b using the chain rule: dL/dw = (2/N) * sum((y_pred - y_true) * x), dL/db = (2/N) * sum(y_pred - y_true). Explain this as backpropagation through the computational graph.

3. Implement SGD training

Initialize w and b (e.g., zeros or small random values). For each epoch, shuffle data, iterate over mini-batches, compute gradients, and update: w = w - lr * dL/dw, b = b - lr * dL/db.

4. Discuss pitfalls and solutions

Address scaling: standardize features to zero mean and unit variance. Discuss divergence: if loss increases, reduce learning rate or use adaptive optimizers. Mention monitoring loss and early stopping.

Key Points to Mention

MSE loss is convex for linear regression, so SGD converges to global minimum with appropriate learning rate.
Gradient derivation via chain rule is analogous to backpropagation in neural networks.
Feature scaling (e.g., standardization) ensures faster convergence and prevents numerical issues.
Learning rate selection is critical: too high causes divergence, too low leads to slow convergence.
SGD updates parameters using mini-batches, which introduces noise but can help escape local minima (though not relevant for convex).
Monitoring loss curve and using techniques like learning rate decay or momentum can improve training stability.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.