TikTok·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

TikTok data scientist interview with a meaty offline marketing measurement question. The whole thing revolved around a single scenario but it had enough layers to keep you busy for a while.

Questions Asked (1)

Pinterest is running a billboard campaign in train stations to drive user traffic and engagement. How would you measure whether the campaign is actually working? Walk through how you'd define and quantify engagement tied to the billboards, estimate who's being exposed to them, and build a framework to forecast incremental gains like sign-ups, sessions, or revenue in the targeted areas.

A/B Testing & ExperimentationProduct Analytics & MetricsProduct Strategy

Author's notes

This was one question but it was basically four questions stacked on top of each other.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining clear success metrics for the billboard campaign, such as incremental sign-ups, sessions, and revenue in targeted areas. Then, outline a measurement framework that includes estimating exposure, setting up geo-based experiments, and using causal inference methods to isolate the campaign's impact. Finally, describe how you would forecast incremental gains by combining exposure estimates with historical conversion rates and running pilot tests.

Pro tip: Emphasize the importance of a control group and pre/post analysis to avoid confounding factors like seasonality or other marketing efforts. Also, mention using geo-lift testing or synthetic control methods to quantify incrementality.

1. Define Success Metrics

Identify key performance indicators (KPIs) such as incremental sign-ups, sessions, and revenue that directly tie to the campaign's objectives. Ensure these metrics are measurable and aligned with business goals.

2. Estimate Exposure

Use foot traffic data, train station statistics, and billboard placement to estimate the number of unique individuals exposed. Consider frequency and demographics to refine the exposed population.

3. Design Measurement Framework

Set up a geo-based experiment with treatment and control areas, or use quasi-experimental methods like difference-in-differences or synthetic control if randomization isn't possible. Track metrics before, during, and after the campaign.

4. Quantify Incremental Impact

Apply causal inference techniques to isolate the campaign's effect from other factors. Calculate lift in sign-ups, sessions, and revenue by comparing treatment and control groups.

5. Forecast and Validate

Build a forecast model using exposure estimates and observed conversion rates to predict incremental gains. Validate with pilot results and adjust as needed.

Key Points to Mention

Incrementality vs. correlation: focus on measuring the causal impact of the billboards, not just overall traffic.
Geo-based experimentation: use treatment and control regions to isolate the campaign's effect.
Exposure estimation: leverage foot traffic data, station demographics, and billboard visibility to estimate reach.
Causal inference methods: difference-in-differences, synthetic control, or propensity score matching to account for confounders.
Forecasting: combine exposure estimates with historical conversion rates to predict incremental sign-ups, sessions, and revenue.
Validation: run pilot tests or holdout groups to validate forecasts and refine the model.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.