I knew orthogonal regression was a thing but had never actually written out the derivation on the spot.
Start by defining the objective: minimize the sum of squared perpendicular distances from points to the line. Then derive the solution using linear algebra, showing that the optimal line passes through the centroid and its direction is the first principal component of the centered data. Finally, explain the equivalence to PCA by connecting the minimization to maximizing variance along the line.
Pro tip: Emphasize that centering is crucial: without centering, the line may not pass through the mean, and the equivalence to PCA breaks. Also, mention that this is total least squares, contrasting with ordinary least squares which minimizes vertical distances.
State that we want to find the line that minimizes the sum of squared perpendicular distances from each data point to the line. This is known as total least squares or orthogonal regression.
Represent the line as passing through a point (the centroid) with direction vector v. Express the perpendicular distance from a point to the line in terms of v and the centered data.
Show that minimizing the sum of squared perpendicular distances is equivalent to maximizing the variance of the projected data onto v. The solution is the eigenvector of the covariance matrix with the largest eigenvalue.
Explain that PCA on centered data finds the principal components as the eigenvectors of the covariance matrix. The first principal component is exactly the direction that minimizes perpendicular distances, hence the equivalence.
Mention that the regression line passes through the centroid and that the perpendicular distances are minimized. Contrast with ordinary least squares, which minimizes vertical distances and is not equivalent to PCA.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clearly stating the model and assumptions: true relationship Y = βX* + ε, but we observe X = X* + u with classical measurement error. Derive the probability limit of the OLS slope and show it equals β times the reliability ratio, which is less than 1, implying attenuation bias toward zero.
Pro tip: Emphasize that the bias depends on the ratio of noise variance to total variance of X, and mention that with multiple indicators or instrumental variables, you can correct for it—this shows practical awareness beyond the derivation.
Define the true model Y = βX* + ε, the observed X = X* + u, and assume classical measurement error: u is mean-zero, uncorrelated with X* and ε, and homoskedastic.
Write the OLS slope as β_hat = Cov(X,Y)/Var(X). Substitute X = X* + u and Y = βX* + ε to compute Cov(X,Y) and Var(X).
Take plim of β_hat as n→∞. Show that plim β_hat = β * Var(X*)/(Var(X*) + Var(u)) = β * λ, where λ is the reliability ratio.
Since 0 < λ < 1, the OLS estimate is biased toward zero. The bias is larger when measurement error variance is large relative to true variance.
Mention that attenuation bias affects inference and predictions; corrections include instrumental variables, multiple indicators, or known reliability ratios.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Blanked a little on the optional stopping theorem conditions.
First, clearly define the simple random walk and the specific stopping rules (e.g., hitting a boundary, fixed time). Then, use martingale properties and optional stopping theorems to compute expected values, ensuring conditions like bounded stopping times or integrability are met. Finally, justify each result by referencing the appropriate theorem and verifying its assumptions.
Pro tip: Always state the conditions for optional stopping explicitly; many candidates forget to check boundedness or uniform integrability, which can lead to incorrect conclusions. Also, relate the results to practical implications, such as expected duration or exit probabilities, to show deeper understanding.
Clearly specify the simple symmetric random walk (S_n) and the stopping time(s) of interest, such as hitting time to a boundary or a fixed time. State whether the walk starts at 0 or another point.
Recognize that S_n is a martingale, and for simple random walk, S_n^2 - n is also a martingale. These are key for computing expected values under stopping.
Use OST to equate the expected value of the martingale at the stopping time to its initial value, but first verify conditions: bounded stopping time, bounded increments, or uniform integrability.
Solve for the desired expectation, such as E[T] or E[S_T], using the martingale equations. For hitting times, set up equations based on the boundaries.
Explain why the computed values make sense, check edge cases, and discuss any assumptions or limitations. Relate to practical scenarios if relevant.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.