Uber·Data Scientist·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Uber DS interview that went deep into experimentation design for a two-sided marketplace. The whole session was basically one long question about switchback experiments, and it got technical fast.

Questions Asked (1)

Design an experiment to measure the causal effect of showing different ETAs on whether a rider session ends in a completed ride request. Walk through why a standard A/B test doesn't work here, and propose an alternative experimental design.

A/B Testing & ExperimentationProduct Analytics & MetricsTechnical Trade-offs

Author's notes

This took me a while to get through cleanly.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining why a standard A/B test fails here due to interference and non-compliance, then propose a switchback or cluster-randomized design that accounts for these issues. Outline the experiment design, metrics, and analysis plan, emphasizing how to measure the causal effect of ETA display on ride completion.

Pro tip: Highlight that ETAs are dynamic and rider behavior is time-sensitive, so a switchback design with short windows and careful randomization at the city or driver level can balance interference and practicality.

1. Identify the causal question and challenges

Clarify the treatment (different ETA displays) and outcome (session ends in completed ride request). Explain why standard A/B test fails: interference (riders/drivers affect each other), non-compliance (riders may not see ETA), and dynamic environment.

2. Propose an alternative design

Suggest a switchback or cluster-randomized design where treatment assignment varies over time or across geographic areas, reducing interference and allowing causal inference.

3. Define metrics and analysis

Specify primary metric (ride completion rate) and guardrail metrics (cancellation rate, wait time). Outline statistical methods like difference-in-differences or instrumental variables if needed.

4. Address practical considerations

Discuss randomization unit, sample size, duration, and potential biases (e.g., time-of-day effects). Mention how to handle spillover and ensure validity.

Key Points to Mention

Interference between riders and drivers (SUTVA violation)
Non-compliance: not all riders see the ETA or act on it
Switchback design: alternating treatment over time within same unit
Cluster randomization: randomize at city or driver level
Use of instrumental variables or encouragement design
Guardrail metrics to monitor unintended consequences

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.