← Meta Interview Insights

Meta·Data Scientist·Technical Phone Screen·Senior

Senior
May 2026

Summary

Brutal causal inference question for a Meta DS role. The whole thing was basically one giant case study on whether web revenue growth was real or just eating into other channels, and they wanted a full analysis plan top to bottom.

Questions Asked (1)

Q1

You notice that web-sourced revenue is higher in 2026 compared to 2025. Design a full causal analysis to determine whether this growth is genuinely incremental or whether it's mostly cannibalization from other acquisition channels like API and mobile. Cover your identification strategy, model specification, key metrics, power analysis, robustness checks, and decision thresholds.

A/B Testing & ExperimentationProduct Analytics & MetricsPricing & Monetization
Author's notes

This was a lot.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a causal inference question about incrementality, not just correlation. Propose a quasi-experimental design (e.g., geo-based or switchback) to isolate the causal effect of web on total revenue, and outline how you'd measure cannibalization from API and mobile. Then detail the model, metrics, power analysis, robustness checks, and decision thresholds to conclude whether the growth is incremental or cannibalistic.

Pro tip: Emphasize that you'd triangulate multiple methods (e.g., geo experiments, synthetic control, and causal impact) to validate findings, and that you'd pre-register your analysis plan to avoid p-hacking. Also, mention that you'd consider the business context: if API and mobile are strategic channels, cannibalization might be acceptable, but you'd still quantify the net effect.

1. Define the causal question and estimand

Clarify the treatment (e.g., web channel exposure) and outcome (total revenue). Specify the estimand: the incremental revenue attributable to web, accounting for cannibalization from API and mobile.

2. Choose identification strategy

Propose a quasi-experimental design such as a geo-based experiment (e.g., DMA-level) or a switchback experiment if web and mobile are mutually exclusive. Alternatively, use synthetic control or difference-in-differences with a suitable control group.

3. Specify model and metrics

Use a model that estimates the effect of web on total revenue while controlling for API and mobile revenue. Key metrics: incremental revenue, cannibalization rate (change in API/mobile revenue per unit change in web revenue), and net lift.

4. Conduct power analysis and robustness checks

Perform power analysis to determine required sample size or duration. Robustness checks: placebo tests, sensitivity to model specification, and validation with alternative methods (e.g., causal impact).

5. Set decision thresholds and interpret results

Define thresholds for incrementality (e.g., net lift > 0 with 95% confidence) and cannibalization (e.g., cannibalization rate < 20%). Use these to decide whether to scale web or adjust strategy.

Key Points to Mention

  • Cannibalization measurement: compare revenue changes in API and mobile relative to web
  • Identification strategy: geo experiments, switchback, synthetic control, or instrumental variables
  • Model specification: include cross-channel effects, time fixed effects, and seasonality controls
  • Power analysis: account for intra-cluster correlation in geo experiments and minimum detectable effect
  • Robustness: placebo tests, pre-trends, and sensitivity to different control groups
  • Decision thresholds: pre-defined criteria for incrementality and acceptable cannibalization levels

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.