Citadel·Data Scientist·Technical Phone Screen
- You have a very high-dimensional linear regression problem where the data doesn't fit in memory. How do you compute XᵀX and Xᵀy using streaming mini-batches, including an intercept term, and then recover the OLS coefficients and standard errors?
- What are the numerical stability concerns with accumulating XᵀX directly, and how do incremental QR or other online methods compare?
- How does ridge regression change the setup, and how do you incorporate the regularization term λI into the accumulated normal equations?
- How would you checkpoint this computation and parallelize it across multiple machines?
“This is the core of the whole question and I actually felt okay here.”