Meta·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

A technical screen for a Data Scientist role at Meta that went deep fast. The whole session was basically one extended problem about call quality modeling, and it had enough sub-parts that it felt like a mini take-home crammed into a live interview.

Questions Asked (1)

You need to set a maximum participant cap K for group video calls. Given call-level telemetry (participants, quality scores, jitter, packet loss, etc.), propose a statistical method to choose K that maximizes total successful participant-minutes while keeping P95 median MOS score at or above 3.8 and dropped call rate at or below 2%. Walk through: (a) fitting and validating a change-point or segmented regression model of call quality vs participant count, (b) computing uncertainty on K and reporting a 95% CI within plus or minus one participant, (c) whether to output a single global K or dynamic caps by device and network type and when to collapse to a single limit, and (d) how much data you need and how you'd handle heavy-tail outliers.

Product Analytics & MetricsA/B Testing & ExperimentationTechnical Trade-offs

Author's notes

This is a beast of a question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a constrained optimization: maximize total successful participant-minutes subject to P95 median MOS ≥ 3.8 and dropped call rate ≤ 2%. Then propose a segmented regression or change-point model to estimate the relationship between participant count and quality metrics, quantify uncertainty on the optimal K, and decide on global vs dynamic caps based on effect heterogeneity. Finally, address data requirements and robust handling of heavy-tailed outliers.

Pro tip: Emphasize that the cap should be chosen based on the confidence interval of the change point, not just the point estimate, and that you would validate the chosen K with an online A/B test to guard against model misspecification.

1. Define objective and constraints

Formalize the goal: maximize total successful participant-minutes (sum over calls of participants × duration × success indicator) subject to P95 median MOS ≥ 3.8 and dropped call rate ≤ 2%. Clarify that 'successful' means calls that meet quality thresholds.

2. Model quality vs participant count

Fit a segmented regression (change-point) model where MOS and drop rate are piecewise linear in participant count, with a breakpoint K0. Use call-level telemetry, control for confounders (device, network, region), and validate via cross-validation or holdout.

3. Estimate uncertainty on K

Use bootstrap or Bayesian methods to get the distribution of the change point and the optimal K that satisfies constraints. Report a 95% CI for K and ensure it is within ±1 participant; if not, collect more data or refine the model.

4. Decide global vs dynamic caps

Test for heterogeneity in the optimal K across device types and network conditions. If differences are material and stable, output dynamic caps; otherwise, collapse to a single global K for simplicity and ease of enforcement.

5. Determine data needs and handle outliers

Estimate required sample size via simulation or power analysis to achieve the desired CI width. Use robust regression (e.g., Huber loss) or trimming/winsorizing to handle heavy-tailed outliers, and report sensitivity analyses.

Key Points to Mention

Constrained optimization: maximize participant-minutes subject to quality constraints.
Segmented regression / change-point model to identify the participant count where quality degrades.
Bootstrap or Bayesian uncertainty quantification for the change point and optimal K.
Heterogeneity analysis: device type, network type (WiFi vs cellular), region, etc.
Sample size calculation and power analysis for detecting the change point with desired precision.
Robust methods for heavy-tailed outliers: Huber loss, quantile regression, trimming/winsorizing.
Validation via A/B test to confirm the chosen K in production.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.