Meta·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

Meta DS interview with a meaty product analytics case built around their Shopping tab. One question, but it had multiple layers and the second half (the diagnosis scenario) is where I felt like I was improvising more than I should have been.

Questions Asked (1)

Design a comprehensive dashboard for the Shopping tab (organic traffic only). Walk through your primary metrics like GMV, purchases, unique buyers, and PDP click-through rate, your secondary/guardrail metrics like bounce rate and search exits, how you'd define each metric at both the event and user level, what dimensions you'd slice by, and what freshness or SLA requirements you'd set. Then, given a scenario where tab engagement is high but purchase rate is low, describe how you'd diagnose the problem: segmentation approach, funnel drop-off analysis, instrumentation validation, and experiments to distinguish UX issues from supply issues.

Product Analytics & MetricsA/B Testing & ExperimentationRoot Cause Analysis

Author's notes

This is basically two questions stitched together and I didn't pace myself well.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer by first defining the dashboard's purpose and audience, then systematically covering primary and guardrail metrics with clear event/user-level definitions, dimensions, and SLAs. For the diagnostic scenario, walk through a structured root-cause analysis: segment, analyze funnel drop-offs, validate instrumentation, and design experiments to isolate UX vs. supply issues.

Pro tip: Emphasize that metrics should be actionable and tied to business goals; for diagnosis, always start with data quality checks before diving into analysis, as instrumentation issues often masquerade as product problems.

1. Define Dashboard Purpose and Audience

Clarify who will use the dashboard (e.g., product managers, engineers, executives) and what decisions it should inform. This guides metric selection and granularity.

2. Select and Define Metrics

Choose primary metrics (GMV, purchases, unique buyers, PDP CTR) and guardrail metrics (bounce rate, search exits). Define each at event and user levels, ensuring clear formulas and data sources.

3. Specify Dimensions and SLAs

Identify key dimensions for slicing (e.g., device, geography, traffic source) and set freshness/SLA requirements (e.g., hourly updates, daily aggregates) based on stakeholder needs.

4. Diagnose Engagement vs. Purchase Gap

Segment users (e.g., new vs. returning, demographics) and analyze funnel drop-offs from tab view to purchase. Validate instrumentation to rule out tracking errors.

5. Design Experiments to Isolate Causes

Run A/B tests targeting UX changes (e.g., layout, search relevance) and supply-side factors (e.g., product availability, pricing) to distinguish between them.

Key Points to Mention

Event-level vs. user-level metric definitions: e.g., GMV per session vs. per user, purchases per event vs. unique buyers.
Funnel analysis: steps from tab view → search → PDP view → add to cart → checkout → purchase, identifying drop-off points.
Segmentation dimensions: device type, user tenure, geography, traffic source, product category.
Instrumentation validation: check for missing events, duplicate logging, or sampling issues that could skew metrics.
Experiment design: A/B tests with UX variations (e.g., search ranking, PDP layout) and supply variations (e.g., inventory levels, pricing) to isolate causes.
Guardrail metrics: bounce rate and search exits as indicators of engagement quality, with thresholds for alerting.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.