DoorDash·Data Scientist·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

DoorDash data science interview centered on a product analytics case about flexible promotion scheduling. One meaty question that blended metric definition, experiment design, and statistical methodology all at once.

Questions Asked (1)

DoorDash is launching a flexible promotions feature (time-of-day deals, merchant-funded discounts, expanded eligibility). What business goals and success metrics would you define? How would you design and monitor an experiment, use pre-period data in your analysis, and decide between an 80/20 or 50/50 traffic split?

A/B Testing & ExperimentationProduct Analytics & MetricsPricing & Monetization

Author's notes

This was a lot to hold in your head at once.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by tying the feature to DoorDash's marketplace goals—growth, engagement, and profitability—then define a metric tree with primary and guardrail metrics. For the experiment, outline a randomized design with pre-period data for variance reduction (CUPED) and sequential testing, and justify the traffic split based on expected effect size, risk, and business constraints.

Pro tip: Emphasize that the traffic split isn't just statistical—it's a business decision balancing learning speed with the cost of a bad experience, and that pre-period data can dramatically increase sensitivity without increasing sample size.

1. Define Business Goals and Metric Tree

Identify how the feature drives growth (e.g., order frequency, new customer acquisition), engagement (e.g., promotion redemption), and profitability (e.g., incremental profit, ROI). Map these to a metric tree with a primary success metric (e.g., incremental orders) and guardrail metrics (e.g., margin, customer satisfaction).

2. Design the Experiment

Choose randomization unit (e.g., customer or market), define treatment and control, and determine sample size and duration based on minimum detectable effect (MDE) and power. Consider potential interference and novelty effects.

3. Leverage Pre-Period Data

Use pre-experiment data to apply variance reduction techniques like CUPED or stratified randomization to increase power. Also check for pre-existing differences to validate randomization.

4. Choose Traffic Split and Monitor

Decide between 80/20 and 50/50 based on risk tolerance, expected effect size, and business constraints. Monitor leading indicators and guardrails daily, and use sequential testing to allow early stopping if needed.

5. Analyze and Decide

After the experiment, analyze the primary metric with pre-period adjustment, check guardrails, and evaluate heterogeneous treatment effects. Make a launch decision based on statistical significance, practical significance, and business impact.

Key Points to Mention

Metric tree with primary success metric (e.g., incremental orders) and guardrails (e.g., profit margin, customer retention)
Randomization unit and potential interference (e.g., customer-level vs. market-level)
Variance reduction using pre-period data (CUPED, stratified randomization)
Traffic split trade-offs: 80/20 for risk mitigation and faster learning, 50/50 for maximum power
Sequential testing and early stopping rules to balance speed and error rates
Heterogeneous treatment effects (e.g., by customer segment, time of day) to inform targeting

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.