TikTok·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Apr 2026

Summary

TikTok MLE interview for a matching/ranking team. One open-ended ML system design question that I fumbled pretty badly, mostly because retrieval systems aren't really my area.

Questions Asked (1)

In a recommender system's recall stage, K candidates are retrieved and passed to a heavy ranker. Design a model or system that determines K dynamically rather than using a fixed value.

System DesignTechnical Trade-offsProduct Analytics & Metrics

Author's notes

I did not have a good answer for this and kind of knew it in the moment.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a trade-off between recall coverage and ranking cost, then propose a lightweight predictor that estimates the marginal value of additional candidates per user. Design the system to output a dynamic K based on user context, item distribution, and latency budget, and validate with offline simulations and online A/B tests.

Pro tip: Emphasize that dynamic K should be optimized for end-to-end metrics like watch time or CTR, not just recall, and mention that you'd cap K to avoid latency spikes during traffic surges.

1. Define the objective and constraints

Clarify that the goal is to maximize ranking quality (e.g., NDCG, CTR) while respecting latency and compute budgets. Identify key constraints: ranker latency, QPS, and recall stage throughput.

2. Model the marginal gain of additional candidates

Propose a lightweight model (e.g., gradient-boosted trees or a small MLP) that predicts the expected improvement in ranking metrics from increasing K by a certain amount, using features like user activity, item popularity, and recall score distribution.

3. Design the dynamic K selection mechanism

Use the predicted marginal gain to choose K per request, e.g., by solving an optimization that balances gain and cost, or by thresholding the predicted gain. Incorporate a fallback fixed K for cold-start or high-load scenarios.

4. Integrate with the ranking stage and system architecture

Describe how the predictor is trained (offline with logged data) and served (online with low latency), and how K is passed to the recall stage. Ensure the system can handle variable K without introducing bottlenecks.

5. Evaluate and iterate

Outline offline evaluation using replay and simulation, and online A/B testing with metrics like watch time, CTR, and latency. Discuss monitoring and retraining cadence.

Key Points to Mention

Trade-off between recall coverage and ranking cost: more candidates can improve quality but increase latency and compute.
Feature engineering for the predictor: user engagement history, context (time, device), recall score distribution, and item diversity.
Cost-aware optimization: formulate as maximizing utility (e.g., expected watch time) minus cost (e.g., latency penalty).
Latency budget and serving constraints: ensure dynamic K does not violate SLA, use caching or precomputation where possible.
Offline evaluation with counterfactual methods: use logged data to estimate performance of different K values without online deployment.
Online experimentation: A/B test dynamic K vs fixed K, measure both business metrics and system metrics (latency, error rates).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.