I knew the answer is the mean but they wanted the actual derivation, not just the punchline.
Define the objective function as the sum of squared deviations from an unknown value, then differentiate with respect to that value and set the derivative to zero. Solve the resulting equation to show that the minimizing value is the arithmetic mean of the array.
Pro tip: Explicitly state that the second derivative is positive (2n), confirming the critical point is a minimum, and connect this to the mean being the least-squares estimator, which is fundamental in regression and ML loss functions.
Let the array be x₁, x₂, ..., xₙ and the unknown value be c. Define f(c) = Σᵢ (xᵢ - c)² as the sum of squared deviations.
Compute f'(c) = Σᵢ 2(xᵢ - c)(-1) = -2 Σᵢ (xᵢ - c) = -2(Σᵢ xᵢ - n c).
Set f'(c) = 0: -2(Σᵢ xᵢ - n c) = 0 ⇒ Σᵢ xᵢ = n c ⇒ c = (1/n) Σᵢ xᵢ, which is the arithmetic mean.
Compute the second derivative f''(c) = 2n > 0, so the critical point is a minimum. Thus the mean minimizes the sum of squared deviations.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, set up the objective function as the sum of absolute deviations and use subgradients to find the condition for optimality, showing that any median minimizes it. Then, describe an efficient algorithm such as quickselect or the median-of-medians to compute the median in linear time, or mention sorting if simplicity is preferred.
Pro tip: Emphasize that the median is robust to outliers and that the subgradient approach generalizes to other quantiles, which is valuable in many data science applications.
Let f(θ) = Σ |x_i - θ|. Explain that we want to find θ that minimizes this sum.
For each term, the subgradient is sign(x_i - θ) (with the convention sign(0) ∈ [-1,1]). The subgradient of the sum is the sum of these signs.
For θ to be a minimizer, 0 must be in the subgradient. This means the number of points less than θ is ≤ n/2 and the number greater than θ is ≤ n/2, which holds exactly when θ is a median.
Use a linear-time selection algorithm (e.g., quickselect or median-of-medians) to find the median. Alternatively, sort the data in O(n log n) and pick the middle element.
Compare O(n) selection vs O(n log n) sorting; mention that for large n, selection is preferred. Also note that the median is robust to outliers.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by recalling that absolute loss minimization yields the median, then generalize to the 90th percentile using the tilted (pinball) loss. Define the loss, show that its minimizer is the 90th percentile, and discuss practical issues like ties and robustness.
Pro tip: Emphasize that the tilted loss is the standard approach for quantile regression and mention that it is convex but not strictly convex, leading to potential non-unique minimizers. Also, highlight that quantiles are more robust to outliers than the mean but less than the median.
Recall that minimizing the absolute loss E|Y - c| yields the median of the distribution. This sets the stage for generalization.
Introduce the tilted loss (also called pinball loss) for quantile τ: L_τ(y, c) = (y - c)(τ - 1_{y < c}). Explain that it asymmetrically penalizes over- and under-prediction.
Show that the minimizer of E[L_τ(Y, c)] is the τ-th quantile of Y. For τ=0.9, it's the 90th percentile. Provide intuition: the loss balances the probabilities of over- and under-prediction.
Explain that if the distribution has a flat region or point mass at the quantile, the minimizer may not be unique. Any value in the interval of quantiles is a minimizer.
Compare to mean (squared loss) and median (absolute loss). The 90th percentile is robust to outliers on the lower end but can be influenced by extreme upper outliers, though less than the mean.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.