TikTok·Data Scientist·Technical Phone Screen
Aug 2025
TikTok data scientist interview focused entirely on experiment design for a messaging feature, specifically a Quick Reply A/B test. Five questions back to back, all deeply technical, no behavioral fluff. Left feeling like I needed to have read three textbooks I hadn't opened in a year.
- Define the primary north-star metric for a Quick Reply feature launch, including the exact formula, unit of analysis, numerator, denominator, inclusion criteria, and a 24-hour attribution window from tap to reply send.
- Propose at least two guardrail metrics that protect long-term product health during the experiment, and specify the formula, measurement unit, and acceptable movement thresholds for each.
- Overall reply send rate increases but average conversation length drops and complaint rate rises among ads-acquired users. How do you segment and interpret these conflicting signals to avoid Simpson's paradox and decide whether to ship, hold, or iterate?
- Design a metric to detect empty engagement such as accidental taps or replies deleted before sending, and explain how you would implement it using existing telemetry and factor it into the launch decision.
- Lay out a pre-analysis plan for this experiment: how would you pre-register metrics, define stopping rules, and prevent metric fishing? Also explain how you would validate that 'exposed' users actually saw the Quick Reply entry point rather than just being eligible for it.
“This is where I spent the most time and still feel shaky about my answer.” The rest of the author's notes on Data Scientist interview at TikTok, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post