Pinterest·Data Scientist·Technical Phone Screen
- How would you define Brand Lift Study versus Conversion Lift Study in ads measurement, and what are the main sources of bias and variance in each?
- When would a Brand Lift Study and a Conversion Lift Study give you conflicting results, and how would you go about reconciling them?
- Compare Difference-in-Differences and Propensity Score Matching for observational ads data. What are the identification assumptions for each, and what robustness check would you run?
- In a randomized conversion lift holdout, treatment group had 1200 users with 42 conversions and control had 1180 users with 33 conversions. Compute the point lift, the pooled standard error under the null, and the two-sided z-statistic. Is the result significant at the 5% level?
- You have pre/post Purchase Intent survey data for exposed and control groups with given sample means and variances. Derive the DiD estimator, compute an approximate standard error using the four independent variance terms scaled by their sample sizes, and report the t-statistic. Show your formulas.
“I knew the high-level distinction but stumbled when they pushed on specific bias sources.”