Airbnb·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Apr 2026

Summary

ML system design round at Airbnb for an MLE role. The question was meaty and covered a lot of ground, probably the most open-ended design problem I've faced in an interview.

Questions Asked (1)

Design an ML system to identify high-quality listings on Airbnb so they can be ranked higher in search and low-quality ones can be flagged or coached.

System DesignTechnical Trade-offsA/B Testing & Experimentation

Author's notes

The first thing I fumbled was the definition of 'quality' itself.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the business goal: define 'high-quality' in terms of guest satisfaction and booking outcomes, then outline an end-to-end ML system from data collection to deployment. Structure your answer around problem framing, data and labeling, model development, evaluation, and integration with search ranking and coaching, emphasizing trade-offs and experimentation.

Pro tip: Highlight the importance of defining a clear, measurable quality score that aligns with business metrics like bookings and guest satisfaction, and discuss how to handle feedback loops and biases in the data to avoid reinforcing existing rankings.

1. Define Quality and Objectives

Clarify what 'high-quality' means for Airbnb listings, using metrics like review scores, booking conversion, and guest satisfaction. Align with business goals: ranking higher for quality listings and flagging/coaching low-quality ones.

2. Data Collection and Labeling

Identify data sources: listing attributes, host behavior, guest reviews, booking history, and search logs. Define labeling strategy: use proxy labels (e.g., high review scores, low cancellation rates) or human annotations for a subset.

3. Model Development and Evaluation

Choose models (e.g., gradient boosting, neural networks) to predict a quality score. Use offline metrics (AUC, NDCG) and online A/B tests to evaluate impact on bookings and guest satisfaction.

4. Integration with Search and Coaching

Incorporate quality score into ranking algorithm (e.g., as a feature or re-ranking). For low-quality listings, trigger interventions like coaching emails or temporary demotion, and measure effectiveness.

5. Monitoring and Iteration

Set up monitoring for model drift, feedback loops, and fairness. Continuously iterate with new data and experiments to improve quality prediction and business outcomes.

Key Points to Mention

Define a clear, measurable quality score aligned with business metrics (e.g., bookings, guest satisfaction).
Use a combination of offline and online evaluation, including A/B testing, to validate model impact.
Address potential biases and feedback loops, especially since ranking affects future data.
Consider trade-offs between precision and recall for flagging low-quality listings (false positives vs. false negatives).
Integrate with existing search ranking infrastructure and ensure scalability.
Design interventions (coaching, demotion) and measure their effectiveness through experiments.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.