Google·Data Scientist·Technical Phone Screen·Senior

SeniorPrefer not to say

May 2026Remote

Summary

Google DS interview with a product experimentation question centered on Google Maps. One question, pretty open-ended, and it went deeper than I expected once we got into the experiment design details.

Questions Asked (1)

Google Maps is considering a feature that recommends personalized jogging routes. How would you assess whether this is worth building, and how would you design a launch experiment including success metrics, triggering logic, unit of randomization, guardrails, and minimum detectable effect?

A/B Testing & ExperimentationProduct Analytics & MetricsProduct Sense & Ideation

Author's notes

I spent too long on the 'is it valuable' part and then had to rush through the experiment design, which is clearly the meat of the question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by evaluating the strategic fit and potential user value of personalized jogging routes, then outline a rigorous A/B test design covering randomization, metrics, and guardrails. Emphasize how you would measure success and make a data-driven launch decision.

Pro tip: Focus on the trade-off between personalization and privacy, and propose a phased approach: start with a small-scale experiment to validate core assumptions before scaling. Also, consider using existing Google Maps data (e.g., popular routes, user preferences) to inform the feature without overcomplicating the MVP.

1. Assess Product Opportunity

Evaluate if the feature aligns with Google Maps' mission and user needs. Consider market demand, competitive landscape, and potential impact on engagement and retention.

2. Define Hypothesis and Metrics

Formulate a clear hypothesis (e.g., personalized routes increase jogging frequency). Define primary success metrics (e.g., weekly active joggers, routes completed) and secondary metrics (e.g., session duration, satisfaction).

3. Design Experiment

Specify unit of randomization (e.g., user-level), triggering logic (e.g., when user opens jogging mode), and sample size calculation. Include guardrail metrics (e.g., app crashes, privacy complaints) and minimum detectable effect (MDE) based on business impact.

4. Analyze and Decide

Run the experiment, analyze results with statistical rigor, and check for heterogeneous treatment effects. Decide whether to launch, iterate, or abandon based on metrics and guardrails.

Key Points to Mention

Unit of randomization: user-level to avoid contamination and measure individual behavior change.
Triggering logic: when a user initiates a jogging session or opens the jogging feature in Google Maps.
Success metrics: primary (e.g., number of jogging routes completed per user per week), secondary (e.g., user retention, satisfaction scores).
Guardrail metrics: app performance (latency, crashes), privacy concerns (e.g., opt-out rates), and impact on other Maps features.
Minimum detectable effect (MDE): determine based on desired business impact, such as a 5% increase in jogging frequency, and calculate required sample size.
Consider network effects and spillover: jogging routes may affect other users (e.g., crowded paths), so consider cluster randomization if needed.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.