Airbnb·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Apr 2026

Summary

Airbnb ML system design round focused on building a relisting recommendation system. Pretty deep technically and covered more ground than I expected in one session.

Questions Asked (1)

Design a system that decides which previously-viewed listings to re-surface to users, covering spatial indexing, a ranking pipeline, evaluation outputs, and how changes to those outputs affect live metrics.

System DesignProduct Analytics & MetricsA/B Testing & Experimentation

Author's notes

This was a lot to pack into one question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the product goal and constraints, then propose a two-stage architecture: candidate generation using spatial indexing (e.g., geohash or H3) to retrieve nearby previously-viewed listings, followed by a ranking model that scores candidates based on user-listing affinity and recency. Finally, define evaluation metrics and explain how offline changes translate to online A/B tests, emphasizing the feedback loop between model updates and live metrics.

Pro tip: Emphasize the importance of counterfactual evaluation and guardrail metrics to avoid degrading user experience when re-surfacing listings; show awareness that re-surfacing can cannibalize new listing discovery and must be balanced with exploration.

1. Clarify Requirements and Constraints

Ask questions to understand the product context: what is the goal (e.g., increase bookings, engagement), what data is available (view history, location, time), and what are the latency and scale requirements. Define success metrics (e.g., CTR, booking rate) and guardrails (e.g., user satisfaction, diversity).

2. Design Spatial Indexing for Candidate Generation

Propose a spatial index like geohash, H3, or Quadtree to efficiently retrieve previously-viewed listings within a radius of the user's current location. Discuss trade-offs (precision vs. recall, indexing cost) and how to handle dynamic user locations and listing availability.

3. Build the Ranking Pipeline

Outline a machine learning ranking model (e.g., GBDT, DNN) that takes features from user, listing, and context (e.g., time since last view, price, distance) to predict relevance or booking probability. Mention two-stage ranking (coarse then fine) and how to incorporate business rules.

4. Define Evaluation Outputs and Metrics

Specify offline metrics (e.g., NDCG, recall@k) and online metrics (e.g., CTR, conversion rate, revenue). Explain how to set up A/B tests to measure the impact of model changes, including sample size calculation and experiment duration.

5. Connect Offline Changes to Live Metrics

Describe the feedback loop: how offline evaluation predicts online performance, how to monitor for drift, and how to iterate. Discuss potential pitfalls like feedback loops, position bias, and the need for counterfactual logging.

Key Points to Mention

Spatial indexing techniques (geohash, H3, Quadtree) and their trade-offs for candidate generation.
Feature engineering for ranking: recency, frequency, monetary (RFM), user-listing interaction history, contextual features.
Two-stage ranking architecture: candidate generation and ranking, with potential for re-ranking.
Offline evaluation metrics (NDCG, MAP, recall) and online A/B testing methodology.
Guardrail metrics to prevent negative user experience (e.g., diversity, freshness, user satisfaction).
Handling of feedback loops and position bias in logged data.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.