Intuit·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

Senior

Apr 2026

Summary

Intuit ML Engineer loop, second session was a case study focused on recommendation systems. The question was meaty enough that I felt like I was building something real rather than just reciting textbook answers.

Questions Asked (1)

Design an offline personalized promotional product recommendation system that handles delayed and sparse conversion feedback. Walk through each component of your design.

System DesignTechnical Trade-offsData Modeling

Author's notes

The sparse feedback angle is where I spent most of my time and honestly where I fumbled a bit early on.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the business context and constraints (offline, personalized, delayed/sparse conversions) and then walk through a modular system design covering data, features, modeling, and evaluation. Emphasize how you handle delayed feedback via techniques like survival analysis or delayed feedback modeling, and sparse conversions via multi-task learning or auxiliary signals. Conclude with trade-offs and monitoring.

Pro tip: Show awareness that in production, delayed feedback often requires a feedback loop where you iteratively update models as conversions arrive, and that you must guard against bias from early feedback. Mention that you'd start with a simple baseline (e.g., collaborative filtering) and then add complexity only if justified by business impact.

1. Clarify Requirements and Constraints

Ask questions to understand the business goal (e.g., increase conversions), data availability (user features, product catalog, historical conversions), latency requirements (offline batch vs. near-real-time), and definition of delayed/sparse feedback.

2. Data Pipeline and Feature Engineering

Design data ingestion for user interactions (clicks, views) and delayed conversions, ensuring proper time-based joins. Engineer features like user demographics, product attributes, interaction history, and time since interaction to capture delayed feedback.

3. Modeling Approach for Delayed and Sparse Feedback

Choose models that handle delayed feedback (e.g., survival analysis, delayed feedback models, or time-decay weighting) and sparsity (e.g., matrix factorization, multi-task learning with auxiliary tasks like click prediction, or meta-learning). Consider using a two-stage approach: candidate generation and ranking.

4. Training and Evaluation Strategy

Define offline evaluation metrics (e.g., AUC, precision@k, expected conversion) and simulate delayed feedback by truncating data. Use techniques like inverse propensity scoring to correct for bias. Plan for online A/B testing to measure true impact.

5. Deployment and Monitoring

Describe how the model will be served (batch precomputed recommendations or real-time API), how often it will be retrained, and how to monitor performance and feedback delays. Include a feedback loop to incorporate new conversions as they arrive.

Key Points to Mention

Handling delayed feedback: survival analysis, delayed feedback models, time-decay weighting, or waiting windows with imputation.
Handling sparse conversions: multi-task learning (e.g., joint click and conversion prediction), matrix factorization with side information, or using auxiliary signals like add-to-cart.
Offline evaluation challenges: need to simulate delayed feedback, use of counterfactual evaluation, and importance weighting.
Personalization techniques: collaborative filtering, content-based filtering, or deep learning models like two-tower architectures.
Trade-offs: batch vs. real-time, model complexity vs. interpretability, and cold-start problem.
Business impact: aligning model metrics with business KPIs (e.g., revenue, conversion rate) and considering long-term effects.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.