Start by clearly stating the logistic regression model and its probabilistic interpretation, then derive the log-likelihood and gradient step-by-step. Emphasize the chain rule and matrix notation, and finish by explaining how gradient descent iteratively updates weights to minimize the loss.
Pro tip: Mention that the gradient of the log-loss has a simple form (predictions minus labels times features), which makes it efficient to compute and interpret. Also note that while gradient descent is standard, for logistic regression you can use more advanced optimizers like L-BFGS, but the underlying gradient is the same.
Define the logistic regression model: linear combination of features passed through a sigmoid function to output probabilities. State the sigmoid function and the probabilistic interpretation for binary classification.
Introduce the negative log-likelihood (log loss) as the objective to minimize. Write it for a single example and then for the entire dataset, highlighting its convexity.
Derive the gradient of the loss with respect to the weights using the chain rule. Show that the gradient simplifies to the sum over examples of (prediction - true label) times feature vector.
Explain the gradient descent algorithm: initialize weights, compute gradient, and update weights in the opposite direction of the gradient scaled by the learning rate. Write the update rule in both scalar and matrix form.
Briefly discuss convergence criteria, learning rate selection, and mention that stochastic or mini-batch gradient descent can be used for large datasets.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.