Meta·Data Scientist·Technical Phone Screen·Intermediate

Intermediate

Jun 2026

Summary

Meta data scientist interview that leaned heavily into stats fundamentals around count data and hypothesis testing. One question but it had a lot of moving parts and the follow-ups came fast.

Questions Asked (1)

Given the full distribution of comment counts per article on a content platform, what summary statistics would you report and why? When does the mean become a bad representation of the data compared to the median? How would you identify and handle outliers in this kind of count data? And if a product team wants to know whether a UI change actually moved engagement, what statistical test would you use?

Product Analytics & MetricsA/B Testing & Experimentation

Author's notes

This felt like four questions duct-taped together and I kind of answered them in the wrong order.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a right-skewed count distribution, then systematically address each sub-question: summary statistics, mean vs. median, outlier handling, and A/B testing. Emphasize the practical implications for product decisions and connect statistical choices to business context.

Pro tip: Always mention that you would visualize the distribution (e.g., histogram, log scale) before choosing statistics, and that for A/B tests on skewed metrics, you might consider non-parametric tests or bootstrapping, but also check if the metric can be transformed or if you should use a different metric like a ratio or binary indicator.

1. Describe the distribution and choose summary statistics

Explain that comment counts are typically right-skewed and non-negative. Report median, IQR, and percentiles (e.g., 90th, 99th) to capture central tendency and tail behavior, along with mean for comparison.

2. Compare mean vs. median and explain when mean fails

Discuss that the mean is sensitive to extreme values and skewness. When the distribution is highly skewed (e.g., a few viral articles with thousands of comments), the mean overstates typical engagement, while the median better represents the typical article.

3. Identify and handle outliers

Use visual methods (boxplots, scatter plots) and statistical rules (IQR rule, z-scores on log-transformed data) to detect outliers. Decide whether to investigate, cap/winsorize, or model separately, depending on whether outliers are errors or genuine extreme engagement.

4. Design and analyze the A/B test for UI change

Propose a randomized controlled experiment. For the test, consider the distribution: if skewed, use a non-parametric test (Mann-Whitney U) or bootstrapped confidence intervals for the difference in medians or means. Alternatively, use a t-test on log-transformed counts or a Poisson/negative binomial regression if appropriate.

5. Connect to product impact and communicate results

Translate statistical findings into business terms: e.g., 'The UI change increased median comments by X% but had no significant effect on the mean due to outliers.' Discuss practical significance and potential next steps.

Key Points to Mention

Right-skewed distribution of count data and the importance of using median and percentiles over mean.
Mean is not robust to outliers; median is a better measure of central tendency for skewed data.
Outlier detection methods: IQR rule, z-scores, visual inspection; handling: winsorizing, capping, or separate analysis.
For A/B testing skewed metrics: consider non-parametric tests (Mann-Whitney U), bootstrapping, or transformations (log, Poisson).
Practical significance vs. statistical significance; effect size and confidence intervals.
Segment analysis (e.g., by article category or user type) to understand heterogeneous treatment effects.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.