← Two Sigma Interview Insights
I knew the formula for the slope off the top of my head but fumbled explaining why the reciprocal relationship breaks down.
Start by clearly defining the two regression models and their slope estimates in terms of sample moments (sums of squares and cross-products). Then derive the product of the slopes and explain why it equals the squared sample correlation coefficient, which is generally not 1, so the second slope is not the reciprocal of the first. Emphasize that OLS minimizes vertical errors, so reversing the roles of x and y changes the objective and yields different slopes.
Pro tip: Mention that the product of slopes equals R², and that the reciprocal relationship would only hold if R²=1 (perfect linear fit). This shows you understand the geometric interpretation of regression and the asymmetry introduced by the error term.
State the two OLS models: y = α + βx + ε and x = γ + δy + η. Clarify that β is the slope of y on x, and δ is the slope of x on y.
Write β = Sxy / Sxx and δ = Sxy / Syy, where Sxy = Σ(xi - x̄)(yi - ȳ), Sxx = Σ(xi - x̄)², and Syy = Σ(yi - ȳ)².
Multiply β and δ: β * δ = (Sxy / Sxx) * (Sxy / Syy) = Sxy² / (Sxx * Syy) = r², where r is the sample correlation coefficient.
The reciprocal of β would be Sxx / Sxy, but δ = Sxy / Syy. These are equal only if Sxy² = Sxx * Syy, i.e., if r² = 1. In general, r² < 1, so δ ≠ 1/β. The asymmetry arises because OLS minimizes squared errors in the dependent variable, so swapping x and y changes the objective function.
Summarize that the product of slopes is R², which measures the goodness of fit, and that the lack of reciprocity reflects the directional nature of regression and the presence of unexplained variance.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, clarify that the question refers to simple linear regression of Y on X and X on Y, and that the slopes are equal when the correlation coefficient r equals 1 or -1. Then, explain that standardizing both variables (subtracting mean and dividing by standard deviation) ensures the slopes are equal, and the common slope is either 1 or -1 depending on the sign of the correlation.
Pro tip: Mention that in practice, perfect correlation is rare, but this transformation is useful for comparing relationships or when using standardized coefficients. Also, note that the common slope equals the correlation coefficient when variables are standardized.
State that we consider two simple linear regressions: Y on X (slope b1) and X on Y (slope b2). The question asks when b1 = b2.
Recall that b1 = r * (sy/sx) and b2 = r * (sx/sy). Setting them equal yields r = ±1, i.e., perfect linear correlation.
Standardize both X and Y to have mean 0 and variance 1. Then sx = sy = 1, so both slopes equal r, which is ±1 under the condition.
Under perfect correlation, the common slope after standardization is either 1 (if r=1) or -1 (if r=-1).
Note that perfect correlation is rare in practice, but standardization is useful for comparing coefficients. Also mention that the common slope equals the correlation coefficient in standardized units.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Honestly the weakest part of my interview.
Clarify that when both variables are measured with error, the OLS slope is attenuated (biased toward zero) due to errors-in-variables. Then discuss alternatives like instrumental variables, Deming regression, and total least squares, highlighting their assumptions and trade-offs.
Pro tip: Mention that the attenuation bias is proportional to the reliability ratio (signal-to-noise ratio) of the predictor, and that without external information or instruments, the true slope is not identified. This shows deep understanding of identifiability.
State that both x and y are noisy measurements of the same true variable, so we have errors-in-variables. Clarify that OLS assumes the predictor is measured without error.
Explain that OLS slope is biased toward zero (attenuation bias) because the measurement error in x is correlated with the error term. The bias factor is the reliability ratio.
Introduce alternatives: instrumental variables (IV), Deming regression, total least squares (TLS), and simulation-extrapolation (SIMEX). Briefly describe each.
For each alternative, state key assumptions: IV requires valid instruments (correlated with true x, uncorrelated with errors); Deming/TLS require known or estimable error variance ratio; SIMEX requires known measurement error variance.
Emphasize that without additional assumptions or data, the true relationship is not identified. In practice, choose method based on available knowledge and validate assumptions.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Quick follow-up, felt more like a gut-check than a deep question.
First, clarify that the two fitted lines are the OLS regression lines of y on x and x on y, which generally have different slopes and intercepts. Then, explain that they intersect at the point of means and coincide only when the correlation is perfect (±1).
Pro tip: Emphasize that the lines are not interchangeable and that their slopes are related by r²; this shows deep understanding of regression geometry and avoids a common pitfall.
State that one line minimizes vertical errors (y on x) and the other minimizes horizontal errors (x on y).
Explain that they intersect at the sample means (x̄, ȳ) and that the y-on-x line is steeper than the x-on-y line when |r| < 1.
State that the lines coincide if and only if the correlation coefficient r is exactly ±1, meaning all data points lie perfectly on a straight line.
Mention that the difference in slopes illustrates regression to the mean: the y-on-x slope is r * (sy/sx), while the x-on-y slope is (1/r) * (sy/sx).
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Interesting edge case I hadn't thought about before.
Clarify the regression setup: if regressing y on x, adding independent noise to x attenuates the slope toward zero (attenuation bias), while regressing x on y inflates the slope. Explain the intuition using the errors-in-variables model and mention the role of noise variance relative to true x variance.
Pro tip: Mention that the attenuation factor is the reliability ratio (signal variance / total variance), and note that with multiple predictors, measurement error in one can bias others in unpredictable directions—showing depth beyond the simple case.
State that there are two slopes: one from regressing y on x (the usual direction) and one from regressing x on y (the reverse direction).
Explain that when x is measured with independent noise, the OLS slope of y on x is attenuated: it equals the true slope times the reliability ratio, which is less than 1.
Show that the slope of x on y is inflated: it equals the true slope divided by the reliability ratio, which is greater than the true slope.
Express the attenuation factor as Var(x_true) / (Var(x_true) + Var(noise)) and note that the product of the two slopes equals R², which is unaffected by noise in x.
Mention that the bias depends on noise variance, and that with multiple regression, measurement error can bias other coefficients in either direction.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.