I rattled these off in a pretty mechanical order and I think it showed.
Start by listing the Gauss-Markov assumptions clearly, then explain each in plain language and its implications for OLS. Emphasize that these assumptions are about the data-generating process and affect properties like unbiasedness, efficiency, and inference. Conclude by noting that in practice, some assumptions are more critical than others and can be tested or relaxed.
Pro tip: Mention that while the Gauss-Markov assumptions guarantee BLUE, the normality assumption is only needed for small-sample inference; with large samples, asymptotic properties often suffice. This shows depth and practical awareness.
State that the model is linear in parameters: Y = Xβ + ε. Clarify that this does not require linearity in predictors, only in coefficients.
Explain that observations are randomly sampled and no independent variable is a perfect linear combination of others. This ensures identifiability and generalizability.
The error term has expected value zero given any values of the predictors: E(ε|X) = 0. This is crucial for unbiasedness of OLS estimators.
Errors have constant variance (homoskedasticity) and are uncorrelated across observations. This ensures OLS is efficient (minimum variance among linear unbiased estimators).
For small samples, errors are normally distributed, enabling exact t- and F-tests. With large samples, this is less critical due to the central limit theorem.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start with a clear definition of multicollinearity as high correlation among predictors, then explain its practical consequences on regression output: unstable coefficient estimates, inflated standard errors, and unreliable p-values, while noting that predictions may still be fine. Emphasize that it does not bias coefficients but affects interpretability and inference, and mention detection and remedies.
Pro tip: In quant finance, multicollinearity often arises from highly correlated factors; focus on how it affects factor attribution and risk decomposition, and consider using regularization or PCA to stabilize estimates while preserving predictive power.
Explain that it occurs when two or more independent variables in a regression model are highly correlated, meaning they contain similar information about the variance in the dependent variable.
Highlight that multicollinearity does not bias the coefficients but makes them highly sensitive to small changes in the model, leading to unstable estimates with large standard errors.
Discuss how inflated standard errors reduce the t-statistics, causing coefficients to appear statistically insignificant even if they are truly important, and widen confidence intervals.
Bring up tools like correlation matrices, Variance Inflation Factor (VIF), and condition indices to identify multicollinearity.
Outline solutions such as removing or combining variables, using regularization (ridge, lasso), PCA, or partial least squares, and note that predictions may remain unaffected while interpretation suffers.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Went through correlation matrix, VIF with the usual thresholds, condition number, eigenvalue analysis.
Start by explaining that multicollinearity is a condition where predictor variables are highly correlated, leading to unstable coefficient estimates. Then outline a systematic diagnostic process, combining correlation analysis, VIF, and condition number, and emphasize the importance of context and iterative refinement.
Pro tip: At Citadel, they value actionable insights: always connect multicollinearity to model performance and business impact, and mention that you'd consider domain knowledge before dropping variables.
Compute a correlation matrix for all predictor variables and identify pairs with high absolute correlation (e.g., >0.8). Visualize with a heatmap to spot patterns.
Calculate VIF for each predictor; values above 5 or 10 indicate problematic multicollinearity. Investigate which variables contribute to high VIF.
Compute the condition number of the design matrix (or eigenvalues of correlation matrix). A condition number >30 suggests multicollinearity; examine eigenvectors for high loadings.
Check for unstable coefficients (large standard errors, sign flips) across different samples or with slight data perturbations. Compare model performance with and without suspected variables.
If multicollinearity is confirmed, consider removing one of the correlated variables, combining them (e.g., PCA), or using regularization. Re-evaluate diagnostics after changes.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Talked through collecting more data, dropping or combining correlated features, centering variables especially for interaction terms, ridge and LASSO and elastic net, and dimensionality reduction approaches.
Start by defining multicollinearity and its consequences, then systematically present the main options (feature selection, dimensionality reduction, regularization, and data collection) and discuss their trade-offs in terms of interpretability, predictive performance, and computational cost. Emphasize that the choice depends on the goal: inference vs. prediction, and the specific context.
Pro tip: At a firm like Citadel, interviewers value practical judgment: mention that multicollinearity is not always a problem if prediction is the goal, and that you would first check VIF and condition number to quantify its severity before deciding on a remedy.
Briefly explain what multicollinearity is and how to detect it (e.g., correlation matrix, VIF, condition number). Highlight that perfect multicollinearity violates OLS assumptions, while high multicollinearity inflates standard errors.
List the main strategies: (1) remove or combine correlated features, (2) use dimensionality reduction (PCA, PLS), (3) apply regularization (ridge, lasso, elastic net), (4) collect more data or use domain knowledge to create better features.
For each option, discuss trade-offs: feature selection improves interpretability but may lose information; PCA creates orthogonal components but reduces interpretability; regularization introduces bias but reduces variance and handles multicollinearity; collecting more data is ideal but often impractical.
Tie the choice to the modeling goal: if inference/interpretability is key, prefer feature selection or domain-driven combination; if prediction is key, regularization or PCA may be better. Mention that sometimes doing nothing is acceptable if multicollinearity doesn't affect predictions.
Summarize that there is no one-size-fits-all solution; the best approach depends on the data, the model, and the business objective. Suggest a pragmatic path: diagnose severity, try simple fixes first, and validate with cross-validation.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
X'X becomes singular, so you can't invert it, OLS breaks down completely.
Start by explaining the mathematical consequence: perfect collinearity makes X'X singular (non-invertible), so the OLS solution is not unique. Then describe how software typically handles it via generalized inverse (pseudoinverse) or by dropping redundant columns, and mention the practical implications for interpretation and prediction.
Pro tip: Emphasize that while predictions may still be stable, the coefficients become unstable and uninterpretable—this distinction often separates strong candidates from those who only recite definitions.
State that perfect collinearity occurs when one predictor is an exact linear combination of others, leading to linear dependence among columns of X.
Describe that X'X becomes singular (rank deficient), its determinant is zero, and it cannot be inverted, so the normal equations have infinitely many solutions.
Mention that software either uses a generalized inverse (e.g., Moore-Penrose pseudoinverse) to produce a solution, or automatically drops one of the collinear columns (e.g., via QR decomposition with pivoting).
Explain that coefficient estimates are not unique and have inflated variance, but fitted values and predictions may remain unchanged; regularization (ridge) can also be used to stabilize estimates.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.