← Glean Interview Insights

Glean·Data Scientist·Technical Phone Screen·Senior

Senior
Jul 2026

Summary

Glean data scientist interview with a pretty gnarly metric integrity problem. The whole thing centered on MAU calculation breaking down when user IDs get remapped mid-window, which is not the kind of thing you think about until it bites you in production.

Questions Asked (3)

Q1

If a company rehashes all user IDs on a single day T, and your MAU window spans dates both before and after T, how does naively counting distinct user IDs distort the metric?

Product Analytics & MetricsRoot Cause Analysis
Author's notes

This took me a second to visualize.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

First, explain that rehashing IDs changes the identifier for the same user, so the same person appears as two different IDs across the window. Then, describe how naive distinct counting would treat those as two separate users, inflating MAU, and quantify the distortion by considering the proportion of users active both before and after T.

Pro tip: Mention that the distortion is not just inflation—it can also cause undercounting if users are split across the boundary and each half falls below activity thresholds, and suggest using a stable surrogate key or crosswalk to deduplicate.

1. Understand the rehashing event

Clarify that on day T, all user IDs are replaced with new hashed values, so the same user has different IDs before and after T.

2. Define the MAU window

The MAU window spans dates before and after T, meaning it includes user activity recorded under both old and new IDs.

3. Analyze naive distinct counting

Naively counting distinct user IDs will count a user active on both sides of T as two distinct users, inflating the MAU count.

4. Quantify the distortion

The inflation equals the number of users active both before and after T (i.e., the intersection). Estimate this by looking at users active in the days just before and after T.

5. Propose mitigation

Suggest using a stable identifier (e.g., account ID, email) or a mapping table to link old and new IDs, or restrict the window to avoid crossing T.

Key Points to Mention

  • Rehashing changes the identifier for the same user, creating a one-to-many mapping between users and IDs.
  • Naive distinct count treats each ID as a separate user, leading to double-counting for users active on both sides of T.
  • The distortion is proportional to the number of users active both before and after T (the overlap).
  • Inflation can be significant if the MAU window is long and many users are active across the boundary.
  • Mitigation: use a stable surrogate key or join with a mapping table to deduplicate users.
  • Alternative: compute MAU separately for pre-T and post-T periods and combine using set union on a stable key.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

What is the maximum possible MAU overestimate as a percentage, and what is the minimum, relative to the true distinct user count over the window?

Product Analytics & MetricsA/B Testing & Experimentation
Author's notes

Blanked on formalizing this cleanly.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

First, clarify that MAU overestimate depends on how you count distinct users over a window (e.g., calendar month vs. rolling 30 days) and the underlying user activity distribution. Then, derive the maximum and minimum overestimate by considering extreme cases: all users active every day (overestimate = 0%) and users active exactly once per month on different days (overestimate can approach 100% if counting daily uniques summed, but relative to true distinct users over the window, the overestimate is bounded by the number of days in the window minus 1, expressed as a percentage).

Pro tip: Emphasize that the maximum overestimate is not unbounded—it's capped by the ratio of the window length to the minimum activity frequency, and always relate it back to the business impact of metric definitions.

1. Define the metric and window

Clarify what 'MAU' means: distinct users active in a calendar month vs. rolling 30-day window, and how 'true distinct user count' is defined over the same window.

2. Identify sources of overestimation

Overestimation occurs when summing daily unique users (or using a shorter window) instead of counting distinct users over the full window; also consider duplicate counting due to multiple devices or logins.

3. Construct extreme scenarios

For maximum overestimate, assume each user is active on exactly one day per month, all on different days, so daily uniques sum to N*days while true distinct is N, giving overestimate = (days-1)/1 *100%. For minimum, assume all users active every day, so overestimate = 0%.

4. Calculate bounds

Maximum overestimate = (D-1)/1 *100% where D is number of days in window (e.g., 30 days → 2900% if summing daily uniques). Minimum overestimate = 0% when all users are active every day.

5. Discuss practical implications

Note that real-world overestimates are typically much smaller due to repeated user activity, and recommend using distinct counts over the full window to avoid inflation.

Key Points to Mention

  • Definition of MAU: distinct users over a 30-day window vs. sum of daily active users.
  • Maximum overestimate occurs when each user is active on only one day per month, all on different days.
  • Minimum overestimate is 0% when all users are active every day.
  • The maximum overestimate as a percentage is (D-1)/1 *100%, where D is days in the window (e.g., 2900% for 30 days).
  • Real-world overestimates are limited by user activity frequency and typically much lower.
  • Recommend using distinct counts over the full window to avoid overestimation.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

How would you redesign the tracking system or warehouse modeling so that MAU stays accurate even when user IDs change like this?

Data ModelingSystem DesignTechnical Trade-offs
Author's notes

My answer went in a few directions: maintain an ID mapping table as a first-class artifact in the warehouse, use a stable internal surrogate key that never changes even when the external ID does, and version or snapshot identity resolution at ingestion time rather than at query time.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the business definition of MAU and the root cause of ID changes (e.g., account merges, device switches, re-registrations). Then propose a layered identity resolution system that maintains a canonical user ID and uses event-level stitching to deduplicate across IDs, ensuring accurate MAU counts.

Pro tip: Emphasize that any redesign must balance accuracy with cost and latency—propose a hybrid approach where real-time stitching handles recent activity and batch reconciliation corrects historical data, and always validate with backtesting against known ground truth.

1. Clarify requirements and constraints

Define what 'MAU' means for the business (e.g., unique users in a 30-day window) and identify all scenarios causing ID changes (e.g., login from new device, account merge, anonymous-to-registered).

2. Design identity resolution layer

Propose a canonical user ID system that maps multiple identifiers (device IDs, emails, cookies) to a single entity, using deterministic and probabilistic matching with a confidence threshold.

3. Model events with identity stitching

Store raw events with all available identifiers, then create a mapping table that links events to canonical IDs. Use graph-based or rule-based stitching to handle merges and splits.

4. Implement MAU computation with deduplication

Compute MAU by counting distinct canonical IDs active in the period, ensuring that events from different IDs belonging to the same user are deduplicated. Consider incremental updates for real-time accuracy.

5. Validate and monitor

Backtest the new system against historical data with known ID changes, measure accuracy and performance, and set up monitoring for identity resolution quality and MAU anomalies.

Key Points to Mention

  • Identity resolution techniques: deterministic (e.g., email, phone) vs. probabilistic (e.g., device fingerprinting, behavioral signals)
  • Canonical user ID and mapping table design (e.g., slowly changing dimensions, graph databases)
  • Event-level stitching and deduplication logic to avoid double-counting
  • Trade-offs between real-time vs. batch processing, accuracy vs. cost/latency
  • Handling edge cases: account merges, splits, anonymous users, and data privacy (GDPR/CCPA)
  • Validation methods: backtesting, A/B testing, and monitoring identity resolution metrics

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.