LinkedIn·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

LinkedIn DS interview with a meaty causal inference question about a redesign rollout where you can't just run a clean A/B test. The kind of problem that sounds manageable until you're actually in the hot seat trying to remember what a propensity score even does under pressure.

Questions Asked (1)

A new mobile app redesign is released as an opt-in upgrade, so users self-select into the new version. How would you measure the causal impact of the redesign? Walk through your causal inference approach, how you'd build comparable treatment and control groups, what features you'd match or weight on beyond past engagement, and how you'd validate your assumptions.

A/B Testing & ExperimentationProduct Analytics & MetricsTechnical Trade-offs

Author's notes

This one wrecked me a little.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Acknowledge the self-selection bias and propose a quasi-experimental design like propensity score matching or weighting to create comparable groups. Outline the steps: define treatment/control, choose covariates, estimate propensity scores, balance check, and estimate treatment effect. Emphasize validation through sensitivity analysis and falsification tests.

Pro tip: Use negative control outcomes (e.g., metrics that shouldn't be affected) to check for hidden biases, and consider instrumental variables if a natural experiment exists (e.g., random rollout of the opt-in prompt).

1. Define treatment and control groups

Treatment group: users who opted into the redesign. Control group: users who did not opt in but are similar in observable characteristics. Consider using a subset of non-opters who were eligible but didn't opt in.

2. Select covariates for matching/weighting

Include past engagement metrics (e.g., sessions, clicks), demographics, tenure, device type, and behavioral features like feature usage patterns. Also consider time-varying factors and pre-treatment outcomes.

3. Estimate propensity scores and create balanced groups

Fit a model (e.g., logistic regression) to predict probability of opting in. Use matching (nearest neighbor, caliper) or weighting (inverse probability weighting) to balance covariates. Check balance using standardized mean differences.

4. Estimate causal effect and validate assumptions

Use the matched/weighted sample to estimate treatment effect on outcomes (e.g., engagement, retention). Validate assumptions: no unmeasured confounding (sensitivity analysis), positivity, and SUTVA. Conduct falsification tests with negative control outcomes.

5. Interpret and communicate results with caveats

Report effect sizes with confidence intervals, discuss limitations (e.g., unobserved confounders), and suggest next steps like randomized experiments if feasible.

Key Points to Mention

Self-selection bias and the need for causal inference methods
Propensity score matching or inverse probability weighting
Covariates beyond past engagement: demographics, device, tenure, behavioral patterns
Balance checks (standardized mean differences, love plots)
Sensitivity analysis for unmeasured confounding (e.g., Rosenbaum bounds)
Negative control outcomes and falsification tests

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.