Google·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

Google DS interview focused entirely on experiment design for a new Maps feature. One long, layered question that kept branching into sub-problems. Felt like a product sense question wearing a stats costume.

Questions Asked (1)

Google Maps wants to recommend personalized jogging routes. How would you design an A/B test for this feature, including your primary metric, secondary metrics, eligibility and trigger conditions, experimental unit, contamination prevention, guardrails, sample size calculation, and rollout plan?

A/B Testing & ExperimentationProduct Analytics & MetricsProduct Sense & Ideation

Author's notes

This question just kept going.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the feature as a personalization algorithm that suggests jogging routes based on user preferences and past behavior. Then, walk through the A/B test design systematically, covering each required component while emphasizing the user experience and business impact. Conclude with a rollout plan that includes iterative testing and monitoring.

Pro tip: Highlight the importance of defining a clear trigger event (e.g., user opens the app in a jogging context) to avoid diluting the treatment effect, and discuss how to handle network effects or interference between users in shared spaces.

1. Define Hypothesis and Goals

Articulate the hypothesis that personalized jogging routes will increase user engagement and satisfaction. Specify the primary goal (e.g., increase in jogging sessions) and secondary goals (e.g., route completion rate, user retention).

2. Choose Metrics and Experimental Unit

Select a primary metric (e.g., number of jogging routes started per user per week) and secondary metrics (e.g., session duration, user ratings). Define the experimental unit as the user, and discuss why user-level randomization is appropriate.

3. Design Eligibility, Trigger, and Contamination Controls

Specify eligibility criteria (e.g., users who have logged at least one jogging activity in the past month) and trigger conditions (e.g., when a user opens the app and taps 'Jogging'). Address contamination by ensuring users are not exposed to both control and treatment, and consider geographic or social network effects.

4. Calculate Sample Size and Guardrails

Determine sample size based on desired power, significance level, and minimum detectable effect. Establish guardrail metrics (e.g., app crash rate, user complaints) to ensure the feature does not harm the overall experience.

5. Plan Rollout and Analysis

Outline a phased rollout: start with a small percentage, monitor metrics, and gradually increase if results are positive. Plan for statistical analysis, including segmentation and long-term holdout to measure sustained impact.

Key Points to Mention

Primary metric: e.g., number of jogging routes started per user per week; secondary metrics: route completion rate, session duration, user retention, and satisfaction scores.
Eligibility: users who have engaged in jogging activities; trigger: when user initiates a jogging session in the app.
Experimental unit: user-level randomization to avoid contamination; consider network effects if routes are shared.
Contamination prevention: ensure users are not exposed to both variants; use unique user IDs and consider geographic isolation.
Sample size calculation: based on baseline metric, expected lift, power (80%), significance level (5%), and minimum detectable effect.
Guardrails: monitor app performance, user complaints, and other key metrics to avoid negative impact; rollout plan: start with 1% traffic, then scale up gradually.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.