← Meta Interview Insights

Meta·Data Scientist·Technical Phone Screen·Senior

Senior
Apr 2026

Summary

Meta data science interview with a meaty stats/behavioral question about skewed distributions and cohort dynamics. Single question but it had four sub-parts that kept branching into more nuance than I expected.

Questions Asked (1)

Q1

Daily shares per user follow a right-skewed distribution with a long tail. On Day 1, split users into two cohorts based on where they fall in that distribution: around the median versus around the 95th percentile. Over the following two weeks, how would you expect each cohort's average daily shares to change, and what role does regression to the mean play versus actual behavioral differences? Which cohort would show more day-to-day variance and why? How would weekly seasonality show up in the cohort time series and how would you control for it? Finally, outline an approach to test whether these percentile-based cohorts remain stable over time.

Product Analytics & MetricsA/B Testing & ExperimentationRoot Cause Analysis
Author's notes

This one sprawled.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by acknowledging the statistical artifact: selecting users at the median vs. the 95th percentile creates different degrees of regression to the mean, which will dominate short-term changes. Then layer on behavioral differences (e.g., heavy users may have different engagement dynamics) and discuss variance, seasonality, and stability testing. Structure your answer around these components, using concrete examples and statistical reasoning.

Pro tip: Emphasize that regression to the mean is a statistical necessity, not a behavioral change, and that any real behavioral effect must be disentangled from it—this shows you understand selection bias and can design analyses to isolate true effects.

1. Regression to the Mean

Explain that the median cohort is selected near the center of the distribution, so their future averages will be close to the overall mean with little regression. The 95th percentile cohort is selected from the extreme tail, so their future averages will regress downward toward the mean. This effect is strongest immediately after selection and diminishes over time.

2. Behavioral Differences

Discuss whether the 95th percentile users are inherently different (e.g., power users) and might sustain higher shares than average, but still lower than their initial extreme. The median cohort may show stable behavior. Any true behavioral effect must be separated from regression to the mean.

3. Variance and Seasonality

The 95th percentile cohort will have higher day-to-day variance because their sharing behavior is more variable (e.g., driven by viral content). Weekly seasonality will appear as periodic fluctuations; control for it using day-of-week fixed effects, seasonal decomposition, or comparing to a control group.

4. Stability Testing

To test if percentile-based cohorts remain stable, track cohort membership over time (e.g., what fraction of Day 1 95th percentile users are still in the top 5% on Day 14). Use transition matrices, rank correlation, or a persistence model. Compare to a random cohort as baseline.

Key Points to Mention

  • Regression to the mean is a statistical artifact; the 95th percentile cohort will show a larger drop in average shares than the median cohort.
  • Behavioral differences: power users may still share more than average, but their initial extreme is unsustainable.
  • The 95th percentile cohort will have higher day-to-day variance due to more variable sharing behavior.
  • Weekly seasonality can be controlled using day-of-week fixed effects, seasonal decomposition, or difference-in-differences with a control group.
  • Cohort stability can be tested by measuring rank persistence, transition matrices, or the proportion of users remaining in the same percentile bin over time.
  • Distinguish between true behavioral change and regression to the mean by comparing to a matched control or using a regression discontinuity design.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.