This is where it started and I actually felt okay here.
Set up a Bayesian inference problem with two hypotheses: H1 = heads is the biased side (p=2/3), H2 = tails is the biased side (p=1/3). Compute the likelihood of observing 3 heads and 1 tail under each hypothesis, multiply by the equal prior, and normalize to get the posterior probability for H1.
Pro tip: Explicitly state the assumptions (equal priors, independence of flips) and note that the result is sensitive to the prior; in a real data science context, you'd often use a Beta prior to incorporate uncertainty about the bias.
Let H1 be 'heads is the heavier side' (P(heads)=2/3) and H2 be 'tails is the heavier side' (P(heads)=1/3). Since priors are equal, P(H1)=P(H2)=0.5.
Under H1, the probability of 3 heads and 1 tail is C(4,3)*(2/3)^3*(1/3)^1 = 4*(8/27)*(1/3)=32/81. Under H2, it is C(4,3)*(1/3)^3*(2/3)^1 = 4*(1/27)*(2/3)=8/81.
Posterior odds = (32/81)/(8/81) = 4. Since priors are equal, posterior probability for H1 = 4/(4+1) = 4/5 = 0.8.
The result 0.8 means there's an 80% chance heads is the heavier side. Check that the likelihood ratio makes sense: observing 3 heads is 4 times more likely if heads is heavier.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Pretty direct once you have the likelihoods.
First, clarify the two hypotheses and the observed data, then specify the prior distributions and likelihood functions for each hypothesis. Compute the marginal likelihoods by integrating over the parameter space, and finally take the ratio to obtain the Bayes factor. Interpret the result in the context of the experiment.
Pro tip: Always state your priors explicitly and justify them; in industry, using default or weakly informative priors can be defensible, but showing awareness of prior sensitivity will impress interviewers.
Clearly state the null and alternative hypotheses (e.g., H0: no effect vs. H1: effect) and describe the observed data (e.g., conversions, sample sizes).
Choose prior distributions for parameters under each hypothesis and write down the likelihood function for the observed data.
Integrate the product of prior and likelihood over the parameter space for each hypothesis to obtain the marginal likelihood (evidence).
Take the ratio of the marginal likelihoods (H1 over H0) to get the Bayes factor.
Interpret the Bayes factor using guidelines (e.g., Jeffreys' scale) and discuss its practical significance for the A/B test.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clarifying the problem setup: assume a Beta prior for the probability of heads, update it with the observed data to get a posterior distribution, then compute the posterior predictive probability as the posterior mean. Walk through the math concisely, and connect it to how Bayesian methods inform decision-making in A/B testing and product analytics at Zoox.
Pro tip: Emphasize that the posterior predictive probability is not just the posterior mean but the expected probability under the posterior, which accounts for uncertainty. Also, mention that with a uniform prior, the result simplifies to (heads + 1) / (total flips + 2), a common pitfall if you forget the prior.
State that you assume a Beta prior (e.g., Beta(1,1) for uniform) and that flips are independent and identically distributed. Confirm that the goal is to predict the next flip, not estimate the underlying bias.
Given h heads and t tails, the posterior is Beta(α + h, β + t). Explain that this conjugacy makes the calculation straightforward.
The posterior predictive probability of heads is the mean of the posterior distribution: (α + h) / (α + β + h + t). Show the formula and plug in numbers if provided.
Explain that this probability incorporates uncertainty and is useful for making predictions in A/B tests, such as estimating the chance a new feature improves a metric. Mention that it can inform decisions like whether to roll out a change.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by setting up a Beta-Binomial conjugate model for n flips with k heads, derive the closed-form posterior, then apply 0-1 loss to get the MAP decision rule. Finally, compute the Bayes factor for the two hypotheses and simplify it to a power of 2.
Pro tip: Emphasize the conjugacy and the simplicity of the Bayes factor—this shows you understand both the math and its practical implications for A/B testing. Also, mention that the 0-1 loss decision rule is equivalent to choosing the hypothesis with higher posterior probability.
Assume a Beta prior for the probability of heads, and a Binomial likelihood for the observed data. State the prior parameters (e.g., Beta(1,1) or general Beta(α,β)).
Use conjugacy to write the posterior distribution as Beta(α + k, β + n - k). Show the closed-form expression.
Under 0-1 loss, the Bayes decision rule is to choose the hypothesis with the highest posterior probability. For a two-sided test (e.g., p=0.5 vs p≠0.5), compute the posterior probability of each and pick the larger.
Define the null hypothesis H0: p=0.5 and the alternative H1: p≠0.5 (or a specific alternative). Compute the marginal likelihood under each and simplify the ratio to a power of 2.
Explain the decision rule in practical terms and discuss the implications of the Bayes factor being a power of 2, such as the evidence scale.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, clarify the model assumptions (e.g., independent and identically distributed flips, exchangeability). Then explain that if the model treats flips as i.i.d. Bernoulli trials, the order does not matter for the likelihood or the distribution of the number of heads; however, order matters if the model includes dependencies (e.g., Markov chain) or if the statistic of interest is a sequence. Conclude by stating the precise condition under which order is irrelevant.
Pro tip: In A/B testing, order often matters when analyzing sequential data or time-based effects; always check whether the model assumes exchangeability or independence, as this determines if order can be ignored. Mentioning exchangeability demonstrates deeper statistical maturity.
Explicitly state the assumed model for the coin flips, such as independent and identically distributed (i.i.d.) Bernoulli trials with a fixed probability p.
Clarify that order matters if the joint probability of a specific sequence depends on the sequence order, or if the statistic of interest is order-dependent (e.g., runs, streaks).
Show that under i.i.d., the joint probability of any sequence with k heads and n-k tails is p^k (1-p)^{n-k}, which is independent of order. Thus, order does not matter for the likelihood or for the distribution of the number of heads.
Explain that order matters if the model includes dependencies (e.g., Markov chain, conditional probabilities) or if the statistic of interest is order-dependent (e.g., longest run of heads).
Conclude that order does not matter if and only if the flips are exchangeable (e.g., i.i.d.) and the statistic of interest is symmetric (e.g., total number of heads).
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by defining the asymmetric prior and how it differs from a symmetric Beta(1,1) prior. Then explain how the posterior is updated using Bayes' theorem, focusing on how the prior parameters shift the posterior distribution. Finally, discuss the impact on the posterior predictive distribution, emphasizing changes in the predicted probability and uncertainty.
Pro tip: Relate the asymmetric prior to real-world scenarios where prior knowledge is not symmetric, such as A/B testing with historical bias, and mention how this affects decision-making in experiments.
Specify a Beta(a, b) prior where a ≠ b, representing different prior beliefs about the probability of heads. For example, Beta(2, 5) indicates a prior belief that tails is more likely.
After observing data (e.g., h heads and t tails), update the prior to get the posterior Beta(a + h, b + t). Explain how the posterior mean and variance are affected by the asymmetry.
The posterior predictive probability of heads is the posterior mean: (a + h) / (a + b + h + t). Discuss how this differs from the symmetric case and how the prior parameters influence the prediction.
Explain how an asymmetric prior can lead to different conclusions in A/B testing, such as requiring more data to overcome a strong prior, and how it affects the trade-off between bias and variance.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.