Start by clearly stating the LDA assumptions, including the (incorrect) equal covariance assumption, and define the discriminant function. Then derive the linear decision boundary by setting the discriminant functions equal and simplifying to a linear equation in x.
Pro tip: Emphasize that LDA assumes equal covariance, so the decision boundary is linear; if covariances are actually different, QDA would be more appropriate. This shows you understand the trade-off between model complexity and assumptions.
Define the two classes with means μ1, μ2 and common covariance Σ (even though true covariances differ). State that LDA assumes equal covariance.
For class k, the discriminant is δ_k(x) = x^T Σ^{-1} μ_k - (1/2) μ_k^T Σ^{-1} μ_k + log P(C=k). Assume equal priors for simplicity unless specified.
Set δ_1(x) = δ_2(x) and simplify. The quadratic terms x^T Σ^{-1} x cancel, leaving a linear equation: w^T x + b = 0, where w = Σ^{-1}(μ1 - μ2) and b = -(1/2)(μ1 + μ2)^T Σ^{-1}(μ1 - μ2) + log(P(C=1)/P(C=2)).
Explain that using a single covariance when they differ leads to a suboptimal linear boundary; the true boundary would be quadratic (QDA). Mention that LDA may still perform okay if covariances are similar.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by writing the general QDA discriminant function for two classes using class-specific means and covariance matrices. Then set the discriminant functions equal to each other to derive the decision boundary, and simplify to obtain an explicit quadratic equation in x and y. Finally, express the boundary as a scalar equation by expanding the quadratic form and collecting terms.
Pro tip: Emphasize that the decision boundary is quadratic because the covariance matrices differ, and mention that if they were equal, it would reduce to a linear boundary (LDA). This shows deep understanding of the model.
For each class k, write δ_k(x) = -0.5 log|Σ_k| - 0.5 (x - μ_k)^T Σ_k^{-1} (x - μ_k) + log π_k, where x = (x, y)^T.
The decision boundary between class 0 and class 1 is given by δ_0(x) = δ_1(x). Write this equation explicitly.
Expand the quadratic forms (x - μ_k)^T Σ_k^{-1} (x - μ_k) for each class, and subtract the two equations to eliminate common terms.
Group terms into x^2, y^2, xy, x, y, and constant. The resulting equation is of the form ax^2 + by^2 + cxy + dx + ey + f = 0.
Write the final scalar equation clearly, defining each coefficient in terms of the class-specific means, covariances, and priors.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, recall the discriminant functions for LDA and QDA, then plug in the given point x=(1,1) along with the provided class parameters (means, covariance matrices, priors). Compute the discriminant scores for each class under both models, compare them, and assign x to the class with the higher score. Show all arithmetic steps clearly.
Pro tip: Always state the decision rule explicitly: classify to the class with the largest discriminant score. Also, if the covariance matrices are equal, QDA reduces to LDA—mention this to demonstrate deeper understanding.
For LDA: δ_k(x) = x^T Σ^{-1} μ_k - 0.5 μ_k^T Σ^{-1} μ_k + log π_k. For QDA: δ_k(x) = -0.5 log|Σ_k| - 0.5 (x-μ_k)^T Σ_k^{-1} (x-μ_k) + log π_k. Clearly define all terms.
List the provided class means (μ_1, μ_2), covariance matrices (Σ for LDA, Σ_1 and Σ_2 for QDA), and prior probabilities (π_1, π_2). If not given, state assumptions (e.g., equal priors).
Calculate inverses and determinants of covariance matrices, and compute quadratic forms like (x-μ_k)^T Σ^{-1} (x-μ_k) for each class. Show intermediate results.
Plug x=(1,1) into the discriminant functions for each class under LDA and QDA. Compute numerical values for δ_1 and δ_2 for both models.
Compare scores: assign x to the class with the higher discriminant. State the final classification for LDA and QDA, and note if they differ.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
This was the part I found most interesting and also the part where I rambled too long.
Start by contrasting QDA and LDA in terms of bias-variance trade-off, emphasizing that with 60 samples per class, QDA's many parameters (p(p+1)/2 per class) often lead to overfitting unless p is small and classes truly have different covariances. Then propose a regularized QDA that shrinks class-specific covariance estimates toward the pooled covariance, and explain selecting the tuning parameter via cross-validation on a metric aligned with the business goal.
Pro tip: Mention that in practice, you'd also consider shrinkage estimators like those in sklearn's LinearDiscriminantAnalysis (with shrinkage='auto') or a custom implementation, and that you'd validate the chosen model using a proper holdout set to avoid overfitting the validation set.
Explain that LDA assumes equal covariance across classes and estimates a single pooled covariance, while QDA estimates a separate covariance per class, leading to more parameters and higher variance.
With only 60 samples per class, QDA's parameter estimates are unstable unless the number of features p is very small (e.g., p < 5) and the true covariances are markedly different; otherwise LDA's pooled estimate is more reliable.
Define a regularized covariance estimate: Σ_k(α) = α Σ_k + (1-α) Σ_pooled, where α ∈ [0,1] controls the interpolation between class-specific and pooled covariance. Then use these in the QDA discriminant function.
Use cross-validation (e.g., stratified k-fold) to choose α that maximizes a relevant metric such as accuracy, F1, or AUC, ensuring the validation folds are representative of the deployment data.
Evaluate the chosen model on a held-out test set, and inspect the selected α to understand the degree of covariance sharing; also consider the computational cost and scalability for TikTok-scale data.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.