Amazon·Data Scientist·Technical Phone Screen·Intermediate

Intermediate

May 2026

Summary

Amazon data scientist technical screen, one meaty coding question about implementing batch gradient descent for linear regression from scratch. Pretty focused session, no fluff.

Questions Asked (1)

Write Python-style pseudocode for batch gradient descent that minimizes mean-squared error on a linear regression problem, and walk through what each step does.

Algorithms & Data StructuresTechnical Trade-offs

Author's notes

I knew the math going in but froze a little when they asked me to actually write it out step by step with explanations.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly defining the linear regression model and the MSE cost function, then present clean pseudocode for batch gradient descent with explicit initialization, forward pass, gradient computation, and parameter updates. Walk through each step, explaining the purpose and the mathematical intuition, and mention practical considerations like convergence and learning rate.

Pro tip: Emphasize that batch gradient descent computes the gradient using the entire training set, which is computationally expensive but stable; contrast it with stochastic gradient descent to show awareness of trade-offs, and mention that feature scaling can significantly speed up convergence.

1. Define the model and cost function

State the linear regression hypothesis h_θ(x) = θ^T x and the mean-squared error cost J(θ) = (1/2m) Σ (h_θ(x_i) - y_i)^2. Explain that the goal is to find θ that minimizes J(θ).

2. Initialize parameters and hyperparameters

Set initial θ (e.g., zeros or small random values), choose learning rate α, and set convergence criteria (e.g., max iterations or gradient norm threshold).

3. Write the gradient descent loop

For each iteration, compute the predicted values for all training examples, calculate the gradient of J(θ) with respect to each θ_j, and update θ_j simultaneously. Include pseudocode for the loop.

4. Explain the gradient computation

Derive the gradient: ∂J/∂θ_j = (1/m) Σ (h_θ(x_i) - y_i) x_i_j. Explain that this is the average of the errors times the feature values, and that it points in the direction of steepest ascent, so we subtract it.

5. Discuss convergence and practical considerations

Mention how to check for convergence (e.g., cost change below threshold), the impact of learning rate, and the importance of feature scaling. Also note that batch gradient descent can be slow for large datasets.

Key Points to Mention

The cost function is convex for linear regression, so gradient descent converges to the global minimum.
Batch gradient descent uses the entire training set for each update, making it deterministic but computationally heavy per iteration.
The learning rate α controls the step size; too large can diverge, too small can be slow.
Feature scaling (e.g., standardization) helps gradient descent converge faster.
Simultaneous update of all parameters is crucial; do not update one parameter and use it for others in the same iteration.
Convergence can be monitored by tracking the cost function or the norm of the gradient.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.