Upstart·Software Engineer·Technical Phone Screen·Senior

Senior

May 2026

Summary

Interviewed for a Research Scientist role at Upstart and got hit with a stats theory question that looked straightforward on the surface but required you to actually know the math cold.

Questions Asked (1)

In simple linear regression, regressing Y on X gives one slope and regressing X on Y gives another. Why aren't those two slopes simply reciprocals of each other? Walk through the math and explain the geometric intuition.

Technical Trade-offsData Modeling

Author's notes

I knew the formulas going in but the geometric part tripped me up more than I expected.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by writing the two ordinary least squares (OLS) slope formulas, then show algebraically why they are not reciprocals unless the correlation is perfect. Follow with the geometric intuition: OLS minimizes vertical errors, so swapping X and Y changes the objective and the fitted line.

Pro tip: Emphasize that the product of the two slopes equals R², which is at most 1, so the slopes are reciprocals only when the fit is perfect. This shows you understand the deeper statistical relationship and can connect it to practical model evaluation.

1. State the two OLS slopes

Write the formulas for the slope of Y on X (β_YX = Cov(X,Y)/Var(X)) and the slope of X on Y (β_XY = Cov(X,Y)/Var(Y)).

2. Check the reciprocal condition

Multiply the two slopes: β_YX * β_XY = [Cov(X,Y)]² / (Var(X)Var(Y)) = R². Since R² ≤ 1, the product is not 1 unless R²=1, so they are not reciprocals in general.

3. Explain the geometric intuition

OLS minimizes vertical squared errors. Regressing Y on X minimizes vertical distances to the line; regressing X on Y minimizes horizontal distances. These are different objectives, so the best-fit lines differ.

4. Connect to correlation and variance

Note that the slopes depend on the variances of X and Y. If the variances are unequal, the slopes will not be reciprocals even if the correlation is high.

5. Summarize the key takeaway

Conclude that the asymmetry arises because OLS treats the variables asymmetrically: one is the predictor, the other the response. The reciprocal relationship only holds when the relationship is perfectly linear (R²=1).

Key Points to Mention

OLS minimizes vertical (Y) errors, not perpendicular errors.
The product of the two slopes equals the squared correlation coefficient (R²).
Reciprocals only when R² = 1 (perfect linear relationship).
Variances of X and Y affect the slopes; unequal variances lead to non-reciprocal slopes.
Swapping X and Y changes the objective function and the fitted line.
Geometrically, the two regression lines are different unless the data are perfectly collinear.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.