TikTok·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

TikTok data science interview that went deep into statistical inference. One question, but it was a beast. The kind of thing where you think you understand Simpson's paradox until someone asks you to prove it formally and then derive conditions to prevent it.

Questions Asked (1)

If average daily TikTok usage increases separately within both male and female user groups, does the overall average necessarily increase too? Walk through a formal proof or counterexample, the conditions under which the overall average is guaranteed to move in the same direction, how shifts in group composition affect the result, and what diagnostic checks you'd run on real data.

A/B Testing & ExperimentationProduct Analytics & MetricsRoot Cause Analysis

Author's notes

I knew Simpson's paradox existed but building a concrete numerical counterexample on the spot was rougher than expected.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Recognize this as a Simpson's Paradox question: subgroup averages can increase while the overall average decreases if group weights shift. Structure your answer by first stating the answer (no, not necessarily), then providing a formal counterexample with numbers, deriving the weighted-average formula to identify conditions for guaranteed increase, and finally discussing real-data diagnostics like weight stability checks and decomposition.

Pro tip: Tie the math to TikTok's product context: mention that a UI change might boost usage for both genders but disproportionately attract male users (who may have lower baseline usage), dragging the overall average down—this shows you think like a product data scientist, not just a statistician.

1. State the answer and intuition

Clearly answer 'no' and explain the intuition: the overall average is a weighted average of subgroup averages, so changes in group proportions can offset subgroup increases. Use a simple analogy if helpful.

2. Formalize with the weighted average formula

Write the overall average as μ = w_m * μ_m + w_f * μ_f, where w_m and w_f are the proportions of male and female users. Show that even if μ_m and μ_f both increase, μ can decrease if the weights shift toward the group with the lower average.

3. Provide a concrete counterexample

Construct a numerical example: e.g., before: 80% female (avg 60 min), 20% male (avg 40 min) → overall 56 min. After: both groups increase by 5 min (female 65, male 45), but composition shifts to 20% female, 80% male → overall 49 min. Show the calculation.

4. Derive conditions for guaranteed increase

Explain that if group weights remain constant, the overall average must increase. More generally, derive the condition: the weighted sum of increases must exceed the effect of weight shifts. Discuss scenarios where weights change but the overall still increases.

5. Discuss real-data diagnostics

List checks: verify group definitions are stable (no users switching groups), check for composition shifts over time, decompose the overall change into within-group and between-group components (e.g., using a Kitagawa decomposition), and run sensitivity analyses.

Key Points to Mention

Simpson's Paradox and how it applies to weighted averages
The weighted average formula: overall = Σ (weight_i * subgroup_avg_i)
The role of group composition (weights) in driving overall trends
A concrete numerical counterexample with before/after data
Conditions for guaranteed increase: constant weights or sufficiently large within-group increases
Diagnostic checks: weight stability, decomposition (within vs. between), and segment-level monitoring

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.