Google·Data Scientist·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Google data scientist technical screen, heavy on the stats and ML theory side. One question that covered a lot of ground around linear models and regularization. Not a coding round, more of a 'prove it on the whiteboard' situation.

Questions Asked (1)

You fit a linear model on X1 and X2, then refit it using transformed predictors Z1 = X1 minus X2 and Z2 = X1 plus X2. First, prove OLS predictions are identical across both models for any invertible linear transformation, and show how the coefficients map between the two. Then, explain whether ridge or lasso penalties preserve this invariance, with precise conditions. For ridge, write out the solutions and show when predictions stay the same. For lasso, give a concrete counterexample where predictions actually differ.

Technical Trade-offsAlgorithms & Data StructuresData Modeling

Author's notes

This wrecked me a little.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by proving OLS invariance under invertible linear transformations using the normal equations and the fact that the column space is unchanged. Then derive the coefficient mapping β = A^{-T} γ. For penalties, analyze ridge by expressing the objective in terms of the transformed coefficients and showing that the solution is a linear transformation of the OLS solution, hence predictions are identical. For lasso, provide a simple counterexample with two data points where the L1 penalty breaks invariance.

Pro tip: Emphasize that the key to invariance is whether the penalty is applied in a way that respects the linear transformation. For ridge, the penalty is quadratic and thus invariant under orthogonal transformations, but for general invertible transformations, the penalty term changes unless the transformation is orthogonal. For lasso, the L1 penalty is not invariant even under orthogonal transformations because it is not rotationally invariant.

1. Prove OLS invariance

Show that OLS predictions are identical for any invertible linear transformation by noting that the column space of the design matrix is unchanged, so the projection onto that space is the same. Use the normal equations to derive the coefficient mapping.

2. Derive coefficient mapping

Given X = Z A^{-1} (or Z = X A), show that the OLS coefficients satisfy β = A^{-T} γ, where β are coefficients for X and γ for Z. Verify that predictions Xβ = Zγ.

3. Analyze ridge regression

Write the ridge objective in both parameterizations. Show that the ridge solution for Z is γ = (Z^T Z + λ I)^{-1} Z^T y, and for X is β = (X^T X + λ I)^{-1} X^T y. Demonstrate that predictions are identical only if A is orthogonal (i.e., A^T A = I), because then the penalty term is invariant.

4. Analyze lasso regression

Explain that lasso does not preserve invariance even for orthogonal transformations because the L1 penalty is not rotationally invariant. Provide a concrete counterexample with a small dataset and a specific transformation where predictions differ.

5. Summarize conditions

Conclude that OLS is invariant under any invertible linear transformation, ridge is invariant only under orthogonal transformations, and lasso is not invariant in general.

Key Points to Mention

OLS invariance relies on the column space of the design matrix being unchanged under invertible linear transformations.
The coefficient mapping between models is β = A^{-T} γ, where Z = X A.
Ridge regression is invariant under orthogonal transformations because the L2 penalty is rotationally invariant.
For non-orthogonal transformations, ridge predictions differ because the penalty term is not invariant.
Lasso is not invariant even under orthogonal transformations due to the non-rotational symmetry of the L1 penalty.
A concrete counterexample for lasso can use two data points and a simple transformation like Z1 = X1 + X2, Z2 = X1 - X2.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.