TikTok·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

TikTok data scientist interview that went deep into causal inference for ad measurement. One big technical question that covered a lot of ground, felt more like a take-home prompt read aloud than a conversational interview.

Questions Asked (1)

You can't randomize ad exposure because users differ on age, education, and income. Design a causal inference approach to estimate the average treatment effect of ad exposure on conversions, covering DAG construction, propensity score modeling, IPW and doubly robust estimators, diagnostics, sensitivity analysis for unobserved confounding, post-treatment bias avoidance, and variance estimation. Also discuss when diff-in-diff or CUPED would be preferable and what assumptions they require.

A/B Testing & ExperimentationProduct Analytics & MetricsTechnical Trade-offs

Author's notes

This was basically a dissertation prompt compressed into one interview question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as an observational causal inference task, then walk through a principled pipeline: define the estimand and DAG, estimate propensity scores, apply IPW and doubly robust estimators, and validate with diagnostics and sensitivity analysis. Emphasize avoiding post-treatment bias by carefully selecting covariates and discuss when alternative designs like diff-in-diff or CUPED are preferable.

Pro tip: Always connect your causal assumptions to the business context—e.g., explain how unobserved confounding could arise from ad targeting algorithms and how you'd probe it. This shows you think beyond the math and understand TikTok's ad ecosystem.

1. Define the causal question and DAG

Specify the average treatment effect (ATE) of ad exposure on conversions, and construct a DAG that includes confounders (age, education, income), treatment, outcome, and potential unobserved confounders. Clearly state assumptions like consistency, positivity, and no unmeasured confounding.

2. Estimate propensity scores and apply weighting

Model the probability of ad exposure given confounders using logistic regression or machine learning, then compute inverse probability weights (IPW) to create a pseudo-population. Check overlap and trim extreme weights.

3. Implement doubly robust estimation

Combine outcome regression with IPW (e.g., augmented IPW) to get a doubly robust ATE estimate that is consistent if either the propensity or outcome model is correct. Use cross-fitting to avoid overfitting.

4. Diagnose and validate

Assess covariate balance via standardized mean differences, check propensity score overlap, and perform sensitivity analysis for unobserved confounding (e.g., E-value, Rosenbaum bounds). Use bootstrap or influence functions for variance estimation.

5. Consider alternative designs

Discuss when diff-in-diff (requires parallel trends) or CUPED (requires pre-experiment covariate correlation) would be preferable, such as when you have panel data or a pre-period, and note their assumptions.

Key Points to Mention

DAG construction and identification assumptions (consistency, positivity, no unmeasured confounding)
Propensity score modeling and diagnostics (overlap, balance)
IPW and doubly robust estimators (AIPW, TMLE) with cross-fitting
Sensitivity analysis for unobserved confounding (E-value, Rosenbaum bounds)
Avoiding post-treatment bias by excluding variables affected by treatment
Variance estimation via bootstrap or influence functions, and when to use diff-in-diff or CUPED

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.