← Meta Interview Insights

Meta·Data Scientist·Technical Phone Screen·Intermediate

Intermediate
May 2026

Summary

Meta data scientist screen with a Bayesian probability question. Pretty standard stuff if you've seen it before, but the numbers are set up to trip you up if you just go on instinct.

Questions Asked (1)

Q1

A classifier flags potentially bad users. 5% of users are actually bad, and the model correctly identifies users in both classes 95% of the time. If the model predicts someone is bad, what's the actual probability they're truly bad?

Product Analytics & MetricsA/B Testing & Experimentation
Author's notes

The 95% accuracy sounds reassuring until you run the numbers and realize the base rate is doing most of the damage.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Recognize this as a Bayes' theorem problem and set up the confusion matrix with given probabilities. Calculate the posterior probability P(Bad | Predicted Bad) using the formula: P(Bad | Pred) = P(Pred | Bad) * P(Bad) / P(Pred). Clearly state assumptions and interpret the result in context.

Pro tip: Emphasize that despite high accuracy, the low base rate (5%) leads to a low precision, which is crucial for understanding model performance in imbalanced settings. This shows you can connect statistical results to business impact.

1. Define events and given probabilities

Let B = user is bad, P = model predicts bad. Given: P(B) = 0.05, P(P|B) = 0.95, P(not P|not B) = 0.95. Derive P(P|not B) = 0.05.

2. Compute total probability of positive prediction

Use law of total probability: P(P) = P(P|B)P(B) + P(P|not B)P(not B) = 0.95*0.05 + 0.05*0.95 = 0.095.

3. Apply Bayes' theorem

Calculate posterior: P(B|P) = P(P|B)P(B) / P(P) = (0.95*0.05) / 0.095 = 0.5.

4. Interpret the result

The probability is 50%, meaning half of flagged users are actually bad. Discuss implications for precision and potential need for threshold adjustment.

Key Points to Mention

  • Bayes' theorem and conditional probability
  • Base rate fallacy and its impact on precision
  • Confusion matrix concepts: true positive, false positive
  • Precision vs. recall trade-off
  • Class imbalance and its effect on model evaluation
  • Business implications of false positives in user flagging

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.