The MLE derivation itself is straightforward, k/n, anyone who's taken a stats course gets there.
Start by writing the likelihood function for n Bernoulli trials with k successes, then derive the MLE by maximizing the log-likelihood. Next, compute the asymptotic variance using the Fisher information or the known variance of the MLE for a proportion, and mention its practical implications.
Pro tip: Connect the MLE and its variance to real-world ML scenarios, such as estimating click-through rates or model calibration, to show practical understanding beyond the math.
Write the likelihood function L(p) = p^k (1-p)^(n-k) and explain that it represents the probability of observing k successes in n trials.
Take the natural log to get l(p) = k log p + (n-k) log(1-p), then compute the derivative with respect to p and set it to zero.
Solve the equation to find p_hat = k/n, and verify it's a maximum by checking the second derivative or boundary conditions.
Use the Fisher information I(p) = n/(p(1-p)) to get the asymptotic variance of p_hat as p(1-p)/n, or equivalently Var(p_hat) ≈ p_hat(1-p_hat)/n.
Mention that the variance decreases with n, and relate to confidence intervals and hypothesis testing for proportions.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by stating the Beta-Bernoulli conjugacy, then derive the posterior by multiplying the prior and likelihood and recognizing the kernel of a Beta distribution. Finally, compute the posterior predictive probability using the law of total probability or the mean of the posterior, and mention its practical implications.
Pro tip: Emphasize that the posterior predictive is not just the posterior mean but an integral over the posterior, which for Beta-Bernoulli simplifies to (alpha + successes) / (alpha + beta + trials). This shows you understand the distinction between parameter estimation and prediction.
Define the Bernoulli likelihood for the data and the Beta(alpha, beta) prior on p. Mention that the Beta is the conjugate prior for the Bernoulli/binomial likelihood.
Multiply the prior and likelihood, ignore constants, and identify the resulting kernel as a Beta distribution with updated parameters alpha + successes and beta + failures.
Use the law of total probability: integrate the likelihood of the next trial over the posterior. Since the posterior is Beta, the integral yields the mean of the posterior, (alpha + successes) / (alpha + beta + n).
Explain how the prior acts as pseudo-counts, how the predictive probability balances prior and data, and mention potential extensions like hierarchical models or non-conjugate priors.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Standard Wald interval for the frequentist side.
Start by clearly defining the problem: you have binomial data and want to estimate p. For the frequentist approach, use the normal approximation to the binomial to construct a Wald confidence interval. For the Bayesian approach, choose a Beta prior, update to a Beta posterior, and compute the equal-tailed credible interval (or highest posterior density). Walk through the formulas, assumptions, and interpretation, highlighting the differences in philosophy and practical implications.
Pro tip: Emphasize that the Wald interval can perform poorly for small samples or extreme p, while the Bayesian credible interval with a weakly informative prior often has better frequentist properties. Mention that in A/B testing, the choice between them depends on whether you want to incorporate prior knowledge and how you interpret probability.
Define p as the true conversion rate, n as sample size, and X as number of successes. State that X ~ Binomial(n, p) and that you observe x successes.
Compute the sample proportion p_hat = x/n. The standard error is sqrt(p_hat(1-p_hat)/n). The 95% CI is p_hat ± 1.96 * SE. Discuss assumptions: np_hat ≥ 10 and n(1-p_hat) ≥ 10 for validity.
Choose a Beta(α, β) prior (e.g., Beta(1,1) for uniform). The posterior is Beta(α + x, β + n - x). Compute the 2.5th and 97.5th percentiles of this Beta distribution to get the equal-tailed credible interval.
Explain that the confidence interval is a frequentist interval: 95% of such intervals cover p in repeated sampling. The credible interval is a Bayesian interval: given the data and prior, there is a 95% posterior probability that p lies in the interval. Discuss practical differences, especially for small samples or extreme proportions.
Mention that the normal approximation is simple but can be inaccurate for small n or p near 0 or 1. The Bayesian approach allows incorporating prior knowledge and provides a direct probability statement, but requires choosing a prior. In A/B testing, Bayesian methods are often preferred for their interpretability and ability to handle sequential analysis.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by defining the two main types of intervals (confidence intervals and credible intervals) and their theoretical guarantees. Then discuss how sample size affects their reliability, emphasizing the asymptotic behavior and practical considerations. Finally, compare and contrast frequentist and Bayesian perspectives, highlighting when each is more appropriate.
Pro tip: Mention that in large samples, frequentist and Bayesian intervals often coincide numerically, but their interpretations differ fundamentally—this shows deep understanding. Also, note that Bayesian intervals can be reliable even with small samples if priors are well-calibrated, which is crucial for online experimentation.
Explain that frequentist confidence intervals have coverage probability guarantees over repeated sampling, while Bayesian credible intervals provide posterior probability statements about the parameter given the data and prior.
Describe how confidence intervals rely on asymptotic normality (CLT) for validity; with small samples, they may undercover unless using exact methods (e.g., t-intervals, bootstrap). Larger samples tighten intervals and improve coverage.
Explain that Bayesian credible intervals are valid for any sample size if the model and prior are correct; as sample size grows, the likelihood dominates the prior, and intervals converge to frequentist ones. Small samples require careful prior selection.
Highlight that frequentist intervals are reliable when assumptions hold and samples are large; Bayesian intervals are reliable when priors are justified and can be more informative in small samples. In A/B testing, both can be used, but Bayesian methods allow direct probability statements.
Suggest that for large-scale experiments, both approaches yield similar results; for small samples or sequential testing, Bayesian methods with informative priors may be preferable. Emphasize checking assumptions and using simulations to validate interval coverage.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.