Uber·Data Scientist·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Uber DS interview with a meaty product experiment question about a rider-incentive program. The whole thing was one big case and they kept pushing for more depth on every layer.

Questions Asked (1)

Design an experiment to measure the impact of a new rider-incentive program. Walk through your metrics for riders, drivers, and overall marketplace matching quality, what else you'd add and why, and how you'd measure effects without knowing much about how the feature works internally.

A/B Testing & ExperimentationProduct Analytics & MetricsProduct Sense & Ideation

Author's notes

This one sprawled in a way I wasn't ready for.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the goal of the rider-incentive program and the experiment design (e.g., randomized controlled trial). Then outline key metrics for riders, drivers, and marketplace matching quality, and explain how you'd measure them without internal knowledge by focusing on observable outcomes and using proxy metrics. Finally, discuss additional metrics and potential pitfalls.

Pro tip: Emphasize the importance of guardrail metrics to ensure the incentive doesn't cannibalize other parts of the business, and discuss how to detect novelty effects and long-term impact.

1. Clarify Objective and Design

Clarify the program's goal (e.g., increase rider retention) and propose a randomized experiment with treatment and control groups, ensuring proper randomization and sample size.

2. Define Metrics

Define primary and secondary metrics for riders (e.g., retention, frequency), drivers (e.g., utilization, earnings), and marketplace (e.g., match rate, wait time, ETA).

3. Measurement Without Internal Knowledge

Use observable data like rider/driver behavior and marketplace outcomes; employ proxy metrics (e.g., incentive redemption rate) and difference-in-differences if randomization isn't possible.

4. Additional Metrics and Guardrails

Include metrics like rider satisfaction (CSAT), driver satisfaction, and guardrails (e.g., cost per incremental ride, cannibalization of other incentives).

5. Analysis and Iteration

Analyze results with statistical tests, check for novelty effects, and consider long-term holdout groups to measure sustained impact.

Key Points to Mention

Randomized controlled trial (A/B test) with proper randomization and power analysis
Rider metrics: retention, frequency, incentive redemption rate, CSAT
Driver metrics: utilization, earnings per hour, acceptance rate, driver satisfaction
Marketplace metrics: match rate, wait time, ETA, fill rate, surge pricing
Guardrail metrics: cost per incremental ride, cannibalization, driver/rider safety
Methods to measure without internal knowledge: proxy metrics, difference-in-differences, intent-to-treat analysis

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.