Bytedance·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

May 2026

Summary

Bytedance ML engineer interview with a meaty system design question around recommendation systems. The whole thing was a single deep-dive, no warmup, just straight into the weeds.

Questions Asked (1)

In a recommendation system, the recall stage typically retrieves a fixed number of K candidate items to pass to the ranking stage. How would you design a system that determines K dynamically per request instead of using a static value? Cover input features, modeling approach, training signals, and how you'd handle serving constraints and evaluation.

System DesignTechnical Trade-offsA/B Testing & Experimentation

Author's notes

This one took me a minute to even parse what they were actually asking.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Frame the problem as a cost-benefit trade-off: dynamically choosing K to maximize the marginal utility of additional candidates while respecting latency and compute budgets. Propose a lightweight predictive model that estimates the optimal K per request using contextual features, trained on offline signals like marginal gain in ranking metrics, and validated via online A/B tests.

Pro tip: Emphasize that the dynamic K model must be extremely cheap to serve—often a simple MLP or gradient-boosted tree on a few features—because it sits in the critical path; also, consider bucketing K into discrete values to simplify serving and experimentation.

1. Define the objective and constraints

Clarify that the goal is to maximize the utility of the final ranked list (e.g., CTR, watch time) minus the cost of retrieving and ranking extra candidates, subject to latency and compute budgets. Establish that K should adapt to request context and item pool characteristics.

2. Identify input features

List features that influence the optimal K: user activity level, historical engagement, query/context signals (time, device), item pool size and diversity, and system load. These features should be cheap to compute at serving time.

3. Design the modeling approach

Propose a two-stage approach: first, estimate the marginal gain curve of adding more candidates (e.g., via a learned model that predicts ranking metric improvement as a function of K), then select K that maximizes net utility. Use a lightweight model (e.g., MLP, GBDT) that outputs either K directly or parameters of the gain curve.

4. Define training signals and labels

Train on offline logs by simulating different K values and measuring the resulting ranking quality (e.g., NDCG, CTR) and cost. Use counterfactual or off-policy evaluation to avoid bias. Alternatively, use online exploration (e.g., epsilon-greedy) to collect data on varying K.

5. Handle serving constraints and evaluation

Ensure the model is served with low latency (e.g., precomputed features, small model). Bucket K into discrete values to simplify serving and A/B testing. Evaluate via online A/B tests measuring both business metrics and system metrics (latency, CPU). Use interleaving or switchback experiments if needed.

Key Points to Mention

Trade-off between recall quality and computational cost: more candidates improve ranking but increase latency and resource usage.
Feature engineering: user engagement history, context, item pool size, and system load as predictors of optimal K.
Modeling the marginal gain: predict the incremental benefit of each additional candidate, then choose K where marginal benefit equals marginal cost.
Training with counterfactual evaluation: use logged data with propensity scores or run online exploration to learn the effect of K.
Serving efficiency: use a lightweight model, precompute features, and consider discrete K buckets to reduce overhead.
Evaluation: offline metrics (e.g., NDCG@K, recall@K) and online A/B tests measuring business metrics and system performance.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.