Start by clarifying requirements and scale, then walk through the end-to-end ML system: data collection, feature engineering, model training, serving, and evaluation. Emphasize trade-offs at each stage, such as batch vs. real-time features, model complexity vs. latency, and offline vs. online metrics.
Pro tip: Uber Eats operates in a two-sided marketplace, so your design must balance user preferences with restaurant supply and delivery logistics. Explicitly discuss how you'd handle cold-start for new users/restaurants and incorporate real-time signals like current wait times.
Ask about scale (users, restaurants, orders), latency requirements, and business goals (e.g., increase order completion, user satisfaction). Define success metrics like CTR, conversion rate, or delivery time.
Describe data sources (user order history, restaurant menus, location, time, ratings) and how to process them. Cover batch (e.g., Spark) and streaming (e.g., Kafka, Flink) pipelines for features like user cuisine preferences, restaurant popularity, and real-time context.
Choose a model architecture (e.g., two-tower neural network for candidate generation, gradient boosted trees for ranking). Discuss training data, negative sampling, and how to incorporate business rules (e.g., promote local restaurants).
Design a low-latency serving architecture: precompute embeddings, use a feature store for online features, and deploy models via a model server (e.g., TensorFlow Serving). Discuss caching, fallbacks, and A/B testing infrastructure.
Define offline metrics (e.g., NDCG, recall@k) and online metrics (CTR, conversion). Plan for A/B tests, monitoring for drift, and feedback loops to retrain models.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
They pushed on this right after I laid out the main architecture.
Start by clarifying the two cold-start scenarios (new restaurants and new users) and their unique challenges. Then propose a multi-pronged strategy that leverages content-based features, contextual information, and exploration-exploitation techniques, while emphasizing evaluation and iteration. Conclude with trade-offs and how you would measure success.
Pro tip: Show awareness of Uber's two-sided marketplace: new restaurants need exposure to generate data, while new users need personalized recommendations without history. Mention using geolocation and time-of-day as strong signals for both.
Ask clarifying questions to understand the scale, available data, and business goals. Distinguish between new restaurants (item cold-start) and new users (user cold-start).
For new restaurants, use metadata (cuisine, price, location, reviews) to compute similarity to existing restaurants. For new users, use contextual signals (location, time, device, signup info) to infer preferences.
Use bandit algorithms (e.g., epsilon-greedy, Thompson sampling) to balance showing new items to gather feedback and showing known good items. For new users, start with popular or diverse recommendations and quickly adapt.
Design the system to rapidly incorporate user interactions (clicks, orders, ratings) to update embeddings. Use transfer learning from similar users or restaurants to bootstrap models.
Define metrics (CTR, conversion, diversity, coverage) and run A/B tests. Monitor for biases and ensure new restaurants get fair exposure. Iterate based on results.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by acknowledging the training-serving skew problem and its impact on model performance. Then describe a systematic approach: define a shared feature schema, use a feature store to serve consistent features, and implement monitoring and validation to detect drift. Emphasize collaboration between ML and platform teams to maintain consistency.
Pro tip: Highlight the importance of logging serving-time features and comparing them to training features in real-time; this not only catches skew but also provides data for retraining. Mention that at Uber, this is critical for dynamic pricing and ETA models where real-time accuracy is paramount.
Establish a single source of truth for feature definitions, including data types, transformations, and expected ranges. This schema should be shared between training and serving pipelines.
Leverage a feature store that provides consistent feature computation for both offline training and online serving. This ensures that the same logic is applied in both environments.
Set up automated checks to compare feature distributions and statistics between training and serving. Monitor for drift, missing values, and unexpected changes in real-time.
Create pipelines that trigger retraining when significant skew is detected, and have rollback mechanisms to revert to previous model versions if issues arise.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Covered NDCG and recall@k for ranking, MAPE for ETA.
Start by clarifying the system's goal and success metrics, then outline a two-pronged evaluation strategy: offline for rapid iteration and online for causal validation. Structure your answer around metric selection, offline evaluation methods, online experimentation design, and iteration/guardrails.
Pro tip: Emphasize that offline metrics are proxies and can mislead; always validate with online A/B tests, and be prepared to discuss how you'd handle interference or network effects in Uber's marketplace.
Clarify the system's objective and map it to measurable metrics: north-star, secondary, and guardrail metrics. Ensure metrics are aligned with business goals and user experience.
Select appropriate offline datasets (historical, holdout) and evaluation methods (e.g., cross-validation, backtesting). Choose metrics like precision/recall, RMSE, or ranking metrics, and set up a pipeline for rapid experimentation.
Design A/B tests or switchback experiments with proper randomization, sample size, and duration. Define success criteria, guardrails, and monitoring for novelty effects and interference.
Use offline results to filter candidates, then validate top candidates online. Analyze results for statistical significance and practical impact, and iterate based on learnings.
Continuously monitor key metrics post-launch, set up alerts for regressions, and have a rollback plan. Consider long-term effects and holdout groups for sustained evaluation.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Talked through offline batch on Hive, online lookup from Redis or Cassandra, and streaming features for freshness.
Start by clarifying requirements: what features, latency targets, and consistency needs. Then propose a unified pipeline with a feature store that serves both batch training and online serving, ensuring consistency via point-in-time correctness. Discuss trade-offs and how to handle scale and freshness.
Pro tip: Emphasize the importance of avoiding training-serving skew by using the same transformation logic and feature definitions in both paths. Mention Uber's Michelangelo or Feast as examples of feature stores that solve this problem.
Ask about latency SLAs, feature freshness, volume, and consistency requirements. Understand the use cases and constraints.
Propose a feature store that ingests data from batch and streaming sources, computes features, and stores them in both offline (for training) and online (for serving) stores.
Use the same feature computation logic for batch and online, and implement point-in-time correctness for training data to avoid leakage.
Choose appropriate storage (e.g., Redis for online, HDFS/S3 for offline) and precompute features where possible. Discuss caching and partitioning.
Set up monitoring for feature freshness, drift, and latency. Plan for backfilling and versioning of features.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Shorter segment of the interview but still meaty.
Start by clarifying the scope: ETA for riders or drivers, real-time or batch, and data sources. Then outline a high-level architecture covering data ingestion, feature engineering, model training, and serving, and finally discuss offline and online evaluation metrics.
Pro tip: Emphasize the importance of real-time features and the trade-off between model complexity and latency, as Uber's ETA system must serve millions of requests with low latency.
Ask questions to understand the specific use case: rider ETA, driver ETA, or delivery ETA? What is the required latency and scale? What data is available?
Describe how to ingest and process historical and real-time data: trip records, traffic conditions, weather, events, and driver locations. Mention batch processing for training and stream processing for real-time features.
Propose a machine learning model, such as gradient boosted trees or deep neural networks, that predicts ETA based on features like distance, time of day, traffic, and driver behavior. Discuss training and updating the model.
Explain how to serve predictions in real-time with low latency, using a scalable microservice architecture, caching, and fallback mechanisms.
List offline metrics (MAE, RMSE, MAPE) and online metrics (actual vs predicted ETA, cancellation rates, user satisfaction). Discuss A/B testing and monitoring.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
This came up late when the interviewer pushed past the baseline.
Start by defining what drift means for a recommendation model and the metrics you would track (e.g., feature drift, prediction drift, business metrics). Then describe a monitoring system that computes these metrics on a schedule, sets alerts, and triggers investigation or retraining. Finally, discuss retraining frequency based on drift detection, business impact, and cost, emphasizing a data-driven approach.
Pro tip: At Uber, real-time monitoring and automated retraining pipelines are critical due to the scale and dynamic nature of the data. Mentioning specific tools like Kafka for streaming data, Prometheus for metrics, and Kubeflow for pipelines can demonstrate practical experience.
Identify the types of drift (data drift, concept drift) and select metrics to monitor: feature distributions, prediction distributions, and business KPIs like CTR or conversion rate.
Implement a system to compute metrics on a schedule (e.g., hourly/daily) using streaming or batch processing. Use tools like Prometheus for metrics and Grafana for dashboards.
Define thresholds for each metric based on historical variability and business impact. Set up alerts to notify the team when drift exceeds thresholds.
When alerts fire, investigate root causes: check data quality, upstream changes, or shifts in user behavior. Use tools like root cause analysis frameworks.
Decide on retraining frequency: either on a fixed schedule (e.g., weekly) or triggered by drift detection. Automate retraining and deployment pipelines with canary testing.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.