Meta·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Apr 2026

Summary

Meta ML engineer screen where they had me implement linear regression from scratch, closed-form solution and all. Pretty math-heavy for a coding round, but they explicitly said AI assistance was fine which was a nice twist.

Questions Asked (1)

Given a set of (x, y) points, implement linear regression by hand using the closed-form solution that minimizes mean squared error. Derive the gradient of MSE with respect to the weights, set it to zero, and code up the resulting weight expression. Then verify your implementation numerically against gradient descent on a small synthetic dataset.

Algorithms & Data StructuresTechnical Trade-offs

Author's notes

The derivation part was fine, I remembered enough calculus to get through it.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly stating the linear regression model and MSE loss, then derive the gradient with respect to weights, set it to zero, and solve for the closed-form solution. Implement the closed-form solution in code, then implement gradient descent and compare the learned weights on a small synthetic dataset to verify correctness.

Pro tip: Mention that the closed-form solution requires inverting X^T X, which can be numerically unstable; suggest using np.linalg.solve or adding a small regularization term. Also, emphasize that gradient descent should converge to the same solution if the learning rate is appropriate and enough iterations are run.

1. Define model and loss

State the linear model y = Xw + b (or include bias in X) and define MSE as (1/n) * sum((y_pred - y_true)^2).

2. Derive gradient and closed-form solution

Compute the gradient of MSE w.r.t. w, set it to zero, and solve for w = (X^T X)^{-1} X^T y. Show the steps clearly.

3. Implement closed-form solution

Code the closed-form expression using numpy, handling the bias term (e.g., by augmenting X with a column of ones).

4. Implement gradient descent

Code gradient descent to minimize MSE, updating weights iteratively. Choose a learning rate and number of iterations.

5. Verify numerically

Generate a small synthetic dataset, run both methods, and compare the resulting weights (and possibly loss curves) to ensure they match within a tolerance.

Key Points to Mention

Closed-form solution: w = (X^T X)^{-1} X^T y (with bias handled via augmented matrix).
Gradient of MSE: (2/n) * X^T (Xw - y).
Numerical stability: use np.linalg.solve instead of explicit inverse; consider regularization if X^T X is singular.
Gradient descent: update rule w := w - learning_rate * gradient; convergence depends on learning rate and scaling.
Verification: compare weights from both methods; they should be close if gradient descent converges.
Synthetic dataset: generate data with known weights and noise to validate both implementations.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.