Reddit·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Apr 2026

Summary

First round at Reddit for an MLE role, focused entirely on ML fundamentals with a CVR modeling problem. Pretty open-ended, which I wasn't expecting for a round labeled 'fundamentals'.

Questions Asked (1)

How would you design a conversion rate prediction model from scratch?

Product Analytics & MetricsTechnical Trade-offsData Modeling

Author's notes

Spent the first few minutes just trying to scope the problem, which features matter, what counts as a conversion, what the label distribution looks like.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the business objective and defining conversion precisely for Reddit's context (e.g., ad click, signup, or purchase). Then walk through a structured ML lifecycle: data collection, feature engineering, model selection, evaluation, and deployment with monitoring, emphasizing trade-offs and Reddit-specific challenges like sparsity and delayed feedback.

Pro tip: Show awareness of Reddit's unique data landscape—anonymized user behavior, community-level signals, and the cold-start problem for new users/ads—and propose pragmatic solutions like hierarchical models or meta-learning. Also, discuss how you'd measure business impact beyond offline metrics (e.g., lift in conversion rate, ROI) to align with product goals.

1. Define the Problem and Success Metrics

Clarify what 'conversion' means for Reddit (e.g., ad click, signup, purchase) and the prediction goal (e.g., probability of conversion for a given user-ad pair). Establish offline and online evaluation metrics (AUC, log loss, calibration, lift, ROI) and constraints (latency, scalability).

2. Data Collection and Feature Engineering

Identify relevant data sources: user demographics, historical behavior, ad content, context (subreddit, time), and community signals. Engineer features like user engagement history, ad relevance, and cross features, handling sparsity and cold-start with embeddings or default values.

3. Model Selection and Training

Choose models based on trade-offs: start with logistic regression for interpretability, then explore gradient boosted trees (XGBoost/LightGBM) for performance, and deep learning (e.g., wide & deep, two-tower) for scalability and complex interactions. Address class imbalance and delayed feedback with techniques like negative sampling or survival analysis.

4. Evaluation and Validation

Use time-based splits to mimic production and avoid leakage. Evaluate with ranking metrics (AUC, precision@k) and calibration. Conduct offline A/B tests or counterfactual evaluation, then online A/B tests to measure true impact.

5. Deployment and Monitoring

Deploy as a real-time service with low latency, using model serving infrastructure. Monitor performance, data drift, and business metrics; set up retraining pipelines and fallback strategies for cold-start or degraded performance.

Key Points to Mention

Handling class imbalance and delayed conversion feedback (e.g., using negative sampling, survival analysis, or time-windowed labels).
Feature engineering for Reddit: user-community interactions, ad creative features, contextual signals (subreddit, time of day), and embeddings for high-cardinality IDs.
Model trade-offs: interpretability vs. performance, latency vs. accuracy, and complexity vs. maintainability.
Cold-start problem for new users/ads and potential solutions like meta-learning, hierarchical models, or content-based features.
Evaluation beyond offline metrics: online A/B testing, uplift modeling, and business impact (ROI, conversion lift).
Scalability and productionization: real-time inference, feature store, monitoring, and retraining pipelines.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.