Meta·Data Scientist·Onsite - Product Sense / Strategy·Senior

Senior

Jun 2026

Summary

Meta DS onsite with a meaty open-ended case on rolling short video into a social newsfeed. Covered metrics, data modeling, and experiment design all in one question, which felt like a lot to juggle in real time.

Questions Asked (3)

How would you define the goal metrics and guardrails for rolling out short video into a social platform's newsfeed?

Product Analytics & MetricsProduct Sense & Ideation

Author's notes

I went straight to engagement and time spent, which felt right, but I fumbled a bit on the guardrail side.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the product goal and the platform's north star metric, then define success metrics that capture both user value and business value. Propose guardrail metrics to monitor potential negative side effects, and outline a phased rollout with A/B testing to validate impact.

Pro tip: Emphasize the importance of leading indicators and counter-metrics to detect unintended consequences early, and suggest a holdout group to measure long-term effects.

1. Clarify Product Goal

Understand the strategic intent behind adding short video: is it to increase engagement, time spent, or attract new users? Align with the platform's overall mission.

2. Define Success Metrics

Choose primary metrics that directly measure the goal, such as daily active users, time spent, or video engagement rate. Ensure they are actionable and aligned with business objectives.

3. Identify Guardrail Metrics

Select metrics to monitor negative impacts, such as user-reported satisfaction, unfollows, hide rates, or declines in other content consumption. These ensure the feature doesn't harm the ecosystem.

4. Design Rollout and Testing Plan

Propose a phased rollout with A/B tests to measure causal impact. Include a holdout group for long-term measurement and define success thresholds for each metric.

5. Monitor and Iterate

Set up dashboards to track metrics in real-time, with alerts for guardrail breaches. Plan for iterative improvements based on data.

Key Points to Mention

North star metric alignment (e.g., meaningful social interactions)
Counter-metrics to detect cannibalization of other content
User satisfaction and well-being metrics (e.g., surveys, sentiment)
Statistical power and sample size considerations for A/B tests
Long-term holdout to measure retention and habit formation
Segmentation by user demographics or behavior to detect heterogeneous effects

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Design a data model for a platform where Content can be text, image, or video, and users interact with that content in various ways.

Data ModelingSystem Design

Author's notes

This part I actually felt decent about.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the platform's scale, content types, and interaction patterns, then propose a flexible schema that separates content metadata from interaction events. Use a combination of relational and NoSQL design principles to handle diverse content and high-volume interactions, and discuss trade-offs for query performance and scalability.

Pro tip: Emphasize how your design supports Meta's key metrics like engagement and time-spent, and mention partitioning strategies for efficient retrieval at scale.

1. Clarify Requirements

Ask about scale, content types, interaction types, and key queries to ensure the design meets business needs.

2. Identify Core Entities

Define main entities: User, Content (with type-specific attributes), and Interaction (e.g., like, comment, share, view).

3. Design Schema

Propose tables/collections for each entity, using polymorphic associations or separate tables for content types, and an event table for interactions.

4. Address Scalability

Discuss partitioning, indexing, and denormalization strategies to handle high write and read volumes.

5. Evaluate Trade-offs

Compare SQL vs NoSQL, normalization vs denormalization, and explain choices based on access patterns.

Key Points to Mention

Polymorphic associations or inheritance for content types (text, image, video)
Event-based schema for interactions (e.g., user_id, content_id, interaction_type, timestamp)
Indexing strategies for common queries (e.g., feed retrieval, engagement metrics)
Partitioning/sharding by user_id or content_id for scalability
Denormalization for read-heavy operations (e.g., precomputed counts)
Handling media metadata (e.g., video length, image resolution) in separate tables or JSON columns

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

How would you measure the impact of introducing short video to the newsfeed, including A/B test setup, user segmentation, and any secondary effects to watch for?

A/B Testing & ExperimentationProduct Analytics & Metrics

Author's notes

Ran through a standard treatment vs control split, talked about randomizing at the user level to avoid spillover.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining the primary success metric (e.g., daily watch time) and guardrail metrics (e.g., user retention, well-being). Then outline a rigorous A/B test design with proper randomization, power analysis, and segmentation. Finally, discuss secondary effects such as cannibalization of other content, novelty effects, and long-term holdouts.

Pro tip: Emphasize the importance of pre-registering your analysis plan and using a long-term holdout to detect novelty effects, as short video often shows initial engagement spikes that fade. Also, mention that you'd monitor for heterogeneous treatment effects across user segments to avoid masking negative impacts on key cohorts.

1. Define Metrics and Hypotheses

Identify the primary metric (e.g., daily watch time per user) and guardrail metrics (e.g., retention, user-reported well-being, content diversity). Clearly state the hypothesis that short video will increase engagement without harming guardrails.

2. Design the A/B Test

Randomize users at the appropriate level (e.g., user-level) to treatment (short video in feed) and control (no short video). Determine sample size via power analysis, set test duration to capture novelty effects, and ensure proper randomization and instrumentation.

3. Segment and Analyze

Pre-define user segments (e.g., by demographics, usage frequency, content preferences) and analyze treatment effects within each segment. Use appropriate statistical methods (e.g., CUPED, bootstrapping) to increase sensitivity and account for multiple comparisons.

4. Monitor Secondary Effects

Track secondary metrics such as cannibalization of other content types, changes in session frequency, and potential negative impacts on well-being. Use long-term holdouts to measure sustained effects beyond the test period.

5. Interpret and Recommend

Synthesize results, considering trade-offs between engagement and guardrails. Provide a clear recommendation on whether to launch, iterate, or abandon, and suggest follow-up experiments if needed.

Key Points to Mention

Primary metric: daily watch time or engagement; guardrail metrics: retention, well-being, content diversity.
A/B test setup: user-level randomization, power analysis, test duration to account for novelty effects.
User segmentation: pre-defined segments (e.g., demographics, usage frequency) and heterogeneous treatment effects.
Secondary effects: cannibalization of other content, novelty effects, long-term holdout to measure sustained impact.
Statistical methods: CUPED, bootstrapping, multiple comparison corrections.
Ethical considerations: monitoring user well-being and avoiding harmful content amplification.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.