State the dot product formula clearly using correct notation: u·v = Σ_{i=1}^n u_i v_i. Then briefly explain the geometric interpretation (u·v = ||u|| ||v|| cos θ) and mention its relevance to ML (e.g., cosine similarity, neural network computations).
Pro tip: Show you understand both the algebraic and geometric definitions, and connect it to practical ML applications like attention mechanisms or similarity metrics. This demonstrates depth beyond rote memorization.
Write the dot product as the sum of component-wise products: u·v = Σ_{i=1}^n u_i v_i. Use clear notation, specifying that u and v are vectors in R^n.
Mention that u·v = ||u|| ||v|| cos θ, where θ is the angle between the vectors. This connects the algebraic definition to geometry.
Note that the dot product is commutative, distributive, and bilinear. Also mention that u·u = ||u||^2.
Explain how the dot product is used in ML, such as in cosine similarity, linear regression, neural network layers, and attention mechanisms.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
I second-guessed myself on whether the denominator was the product of the norms or the sum.
State the cosine similarity formula clearly, emphasizing that it is the dot product of u and v divided by the product of their Euclidean norms. Explain that this measures the cosine of the angle between the vectors, making it invariant to magnitude. Optionally, connect it to common ML applications like text similarity or recommendation systems.
Pro tip: Mention that cosine similarity is equivalent to the dot product of L2-normalized vectors, which is why it's often preferred over Euclidean distance for high-dimensional sparse data. Also, note that it ranges from -1 to 1, with 0 indicating orthogonality.
Write the formula: cos(θ) = (u · v) / (||u|| ||v||). Clearly identify the numerator as the dot product and the denominator as the product of the Euclidean norms.
Define u · v as the sum of element-wise products, and ||u|| as the square root of the sum of squares of u's components. Do the same for ||v||.
Explain that the result is the cosine of the angle between u and v, ranging from -1 (opposite) to 1 (identical direction), with 0 indicating orthogonality.
Mention typical uses such as document similarity, recommendation systems, and embedding comparisons, highlighting its robustness to magnitude differences.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
State both formulas clearly, define each variable, and briefly explain the geometric intuition behind each. Then relate them to machine learning contexts, such as loss functions and distance metrics, to show practical understanding.
Pro tip: Mention that Euclidean distance is sensitive to feature scales and often requires normalization, while Manhattan distance is more robust to outliers—this shows awareness of real-world ML implications.
Let the two points be p = (p1, p2, ..., pn) and q = (q1, q2, ..., qn) in n-dimensional space. Clearly state that n is the number of dimensions.
Euclidean distance is the square root of the sum of squared differences: d(p, q) = sqrt(Σ (pi - qi)^2). Explain that it represents the straight-line distance.
Manhattan distance is the sum of absolute differences: d(p, q) = Σ |pi - qi|. Explain that it represents the distance along grid lines.
Highlight that Euclidean is sensitive to large differences due to squaring, while Manhattan treats all differences linearly. Mention that Manhattan is also called L1 or taxicab distance.
Give examples: Euclidean distance is used in KNN, K-means, and as MSE loss; Manhattan distance is used in L1 regularization (Lasso) and robust regression.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clearly defining the Mean Squared Error (MSE) for n predictions versus targets, then systematically apply calculus to derive the partial derivatives with respect to weight and bias. Show each step of the derivation, explaining the chain rule and how the derivatives are used in gradient descent.
Pro tip: Emphasize that the 2/n factor in the gradient simplifies to 2/n * error * input, which is crucial for efficient implementation. Also, mention that these gradients are the foundation of backpropagation in neural networks.
Write the MSE as J(w,b) = (1/n) * sum_{i=1}^{n} (y_i - (w*x_i + b))^2, clearly identifying predictions, targets, weight, and bias.
Apply the chain rule: ∂J/∂w = (2/n) * sum_{i=1}^{n} (y_i - (w*x_i + b)) * (-x_i). Simplify to -2/n * sum (error_i * x_i).
Similarly, ∂J/∂b = (2/n) * sum_{i=1}^{n} (y_i - (w*x_i + b)) * (-1). Simplify to -2/n * sum (error_i).
Explain that these gradients indicate the direction to adjust w and b to minimize MSE, and are used in gradient descent updates: w := w - α * ∂J/∂w, b := b - α * ∂J/∂b.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Clearly state both formulas, define each term, and briefly explain when to use each method. Emphasize that min-max normalization scales to a fixed range (usually [0,1]) while z-score standardization centers data around mean 0 with standard deviation 1.
Pro tip: Mention that min-max normalization is sensitive to outliers, whereas z-score standardization is more robust; also note that the choice depends on the algorithm (e.g., neural networks often prefer standardization, while distance-based methods may benefit from normalization).
State the formula: x' = (x - min(x)) / (max(x) - min(x)). Explain that it rescales features to a fixed range, typically [0,1].
State the formula: x' = (x - μ) / σ, where μ is the mean and σ is the standard deviation. Explain that it transforms data to have mean 0 and standard deviation 1.
Discuss when to use each: min-max for bounded ranges and when no outliers; z-score for algorithms assuming Gaussian distribution or when outliers are present.
Highlight that both methods require fitting on training data and applying the same parameters to test data to avoid data leakage.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
They called it a bonus but it felt like they expected it.
Start by writing the sigmoid function and its derivative, then the softmax function and its derivative, and finally binary cross-entropy. Explain each formula and its significance in neural networks, and mention any numerical stability considerations.
Pro tip: When writing the softmax derivative, clarify that it's a Jacobian matrix and often used in combination with cross-entropy loss, which simplifies the gradient. Also, mention that binary cross-entropy is equivalent to softmax with two classes.
Write the sigmoid function σ(x) = 1/(1+e^{-x}) and its derivative σ'(x) = σ(x)(1-σ(x)). Explain that it squashes inputs to (0,1) and is used for binary classification.
Write the softmax function softmax(z)_i = e^{z_i}/Σ_j e^{z_j} and its derivative ∂softmax(z)_i/∂z_j = softmax(z)_i (δ_{ij} - softmax(z)_j). Explain that it generalizes sigmoid to multi-class and outputs a probability distribution.
Write the binary cross-entropy loss L(y, ŷ) = -[y log(ŷ) + (1-y) log(1-ŷ)]. Explain that it measures the dissimilarity between predicted probabilities and true labels, and is commonly used with sigmoid output.
Mention that implementations often use the log-sum-exp trick for softmax and clip probabilities in cross-entropy to avoid log(0). Also, note that the derivative of softmax is often not computed explicitly when combined with cross-entropy.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.