I knew the punchline but fumbled the derivation.
Start by explaining the intuition behind why the median minimizes L1 and the mean minimizes L2 using simple examples and geometric interpretations. Then provide a rough derivation using calculus (derivative of the loss function) to show the optimality conditions. Finally, extend to 2D with Manhattan and Euclidean distances, and discuss how outliers affect the mean's robustness.
Pro tip: Mention that the median is robust to outliers because it depends only on the order of values, while the mean is sensitive to extreme values. This ties into real-world applications like robust statistics and choosing appropriate loss functions in machine learning.
Explain that for L1, moving the estimate towards a data point reduces the sum of absolute deviations until you hit the median; for L2, the squared penalty makes the mean the balance point.
Show that the derivative of sum |x_i - c| is the sum of signs, which is zero when equal numbers of points are on each side (median). For L2, derivative of sum (x_i - c)^2 is -2 sum (x_i - c), zero when c is the mean.
For Manhattan distance, the optimal point minimizes sum of L1 distances; the solution is the geometric median (not necessarily a data point). For Euclidean distance, it's the centroid (mean).
Explain that the mean is not robust: a single outlier can pull it arbitrarily far, while the median remains stable. This makes the mean a bad estimator when data has outliers or heavy tails.
Discuss when to use each: L1/median for robust estimation, L2/mean for efficiency under Gaussian noise. Mention applications like regression (LAD vs OLS) and clustering (K-medians vs K-means).
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.