I knew the setup well enough to write down the sample covariance and point to the leading eigenvector as the natural estimator.
First, explain that the spike direction is estimated by the top eigenvector of the sample covariance matrix. Then, use the Davis-Kahan theorem to bound the sine of the angle between the estimated and true spike directions, expressing the error in terms of the spectral gap and the perturbation norm. Finally, specialize to the spiked covariance model to derive the error rate as a function of sample size, dimension, and signal strength.
Pro tip: Mention that when the signal strength is below the Baik-Ben Arous-Péché (BBP) threshold, the top eigenvector is asymptotically orthogonal to the true spike, so consistent estimation is impossible. This shows awareness of fundamental limits.
Describe the spiked covariance model: Σ = I + λ uu^T, where u is the spike direction and λ is the signal strength. Estimate u by the top eigenvector of the sample covariance matrix S = (1/n) Σ_{i=1}^n x_i x_i^T.
Apply the Davis-Kahan theorem: sin θ ≤ ||E|| / gap, where E = S - Σ is the perturbation, θ is the angle between estimated and true eigenvectors, and gap is the difference between the top eigenvalue and the rest of the spectrum.
Bound ||E|| using concentration inequalities (e.g., ||E|| ≲ sqrt(d/n) for sub-Gaussian data). Compute the spectral gap: for the spiked model, gap ≈ λ when λ is large, but more precisely gap = λ + 1 - (1 + sqrt(d/n))^2 in the high-dimensional regime.
Combine to get sin θ ≲ sqrt(d/n) / gap. Simplify to obtain the rate: if λ ≫ sqrt(d/n), then sin θ ≲ sqrt(d/n)/λ. Discuss the BBP threshold: if λ < sqrt(d/n), the top eigenvector is not consistent.
Discuss how the error depends on n, d, and λ, and mention that when d/n is not negligible, the effective signal is reduced. Suggest possible improvements like shrinkage or using more sophisticated estimators.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.