Attentive·Data Scientist·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Got a pretty intense diagnostic case question for a Data Scientist role at Attentive. One question, but it was a beast, covering everything from instrumentation sanity checks to causal inference to experiment design. Not a vibe-check round at all.

Questions Asked (1)

Ride volume at a major rideshare platform drops 14% month-over-month and 9% year-over-year after normalizing for days in the month. Walk through how you'd (a) confirm the drop is real and not a data artifact, (b) decompose it into demand-side vs supply-side causes, and (c) quantify the top two root causes. Be specific about metrics, data cuts, statistical methods, and any experiments you'd run to validate your hypothesis.

Root Cause AnalysisProduct Analytics & MetricsA/B Testing & Experimentation

Author's notes

This one took me a while to even organize mentally.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by validating the data pipeline and metric definitions to rule out artifacts, then decompose the drop into demand and supply components using funnel and cohort analyses. Finally, quantify the top two root causes by isolating their impact through statistical methods and experiments, prioritizing actionable insights.

Pro tip: Always tie your analysis to business impact by estimating the revenue or market share implications of the drop, and propose a quick experiment to validate the root cause before deep-diving into complex models.

1. Validate Data Integrity

Check for data pipeline issues, logging errors, or metric definition changes by comparing raw event counts, auditing ETL jobs, and verifying normalization methods. Ensure the drop is not due to missing data or seasonal adjustments.

2. Decompose Demand vs. Supply

Analyze demand-side metrics (e.g., ride requests, app opens, search-to-request rate) and supply-side metrics (e.g., driver availability, acceptance rate, wait times) across dimensions like geography, time, and user segments to identify where the drop originates.

3. Quantify Root Causes

Use statistical methods like regression, difference-in-differences, or causal impact analysis to isolate the effect of each potential cause. Prioritize the top two causes by their contribution to the overall drop.

4. Validate with Experiments

Design and run A/B tests or quasi-experiments (e.g., switchback tests) to confirm causality. For example, test a pricing change or driver incentive program to see if it reverses the drop.

Key Points to Mention

Data validation techniques: anomaly detection, pipeline audits, and metric consistency checks.
Demand-side metrics: ride requests, conversion rates, user retention, and cohort analysis.
Supply-side metrics: driver online hours, acceptance rates, wait times, and geographic heatmaps.
Statistical methods: regression, difference-in-differences, causal impact, and time-series decomposition.
Experiment design: A/B tests, switchback tests, and holdout groups to validate hypotheses.
Business impact: revenue loss, market share, and prioritization based on actionable insights.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.