TikTok·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

May 2026

Summary

TikTok system design round, one big question that basically ate the whole session. The scope was massive and I kept second-guessing whether to go deeper on one piece or keep moving.

Questions Asked (1)

Design a high-traffic service that maintains p99 latency SLOs during sudden traffic spikes. Walk through admission control, token-bucket rate limiting, priority queues, deadlines, and timeouts. Compare different load shedding strategies and explain when to apply each at the load balancer versus the application layer. Also cover circuit breakers, backpressure mechanisms, protecting critical dependencies, and what metrics you'd track to validate everything is working.

System DesignTechnical Trade-offs

Author's notes

This was basically five questions rolled into one and I felt it.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the SLO (e.g., p99 < 200ms) and traffic characteristics, then structure your answer around a layered defense: load balancer-level admission control and rate limiting, application-level priority queues and deadlines, and dependency protection with circuit breakers and backpressure. Emphasize trade-offs between different load shedding strategies and how you'd validate with metrics like p99 latency, error rates, and queue depths.

Pro tip: Tie every mechanism back to the SLO: e.g., 'We shed low-priority requests to keep p99 under 200ms for critical traffic.' This shows you optimize for business impact, not just technical correctness.

1. Clarify requirements and SLOs

Ask about traffic patterns, SLO targets (e.g., p99 latency, availability), and criticality of different request types. Establish what 'high-traffic' means and what spikes look like.

2. Design admission control and rate limiting

Explain token-bucket rate limiting at the load balancer to smooth spikes, and admission control at the application layer to reject excess requests early. Discuss global vs. per-user limits.

3. Implement priority queues and deadlines

Describe how to classify requests by priority (e.g., critical vs. best-effort) and use priority queues to ensure important traffic meets SLOs. Set deadlines and timeouts to avoid wasting resources on stale requests.

4. Protect dependencies with circuit breakers and backpressure

Explain circuit breakers to fail fast when a dependency is unhealthy, and backpressure to slow down producers when downstream is saturated. Discuss protecting critical dependencies with separate pools or bulkheads.

5. Choose load shedding strategies and define metrics

Compare load shedding at LB (e.g., drop connections) vs. app layer (e.g., reject based on priority). Define metrics: p99 latency, error rate, queue depth, circuit breaker state, and throughput, and how to alert on them.

Key Points to Mention

Token-bucket rate limiting: allows bursts up to bucket size, refills at a steady rate; ideal for smoothing spikes at the edge.
Admission control: reject requests early when system is overloaded, using techniques like concurrency limits or queue depth thresholds.
Priority queues: ensure high-priority requests (e.g., payments, critical reads) are served first; use weighted fair queuing or strict priority.
Deadlines and timeouts: propagate deadlines through the call graph; cancel work when deadline exceeded to free resources.
Load shedding strategies: at LB, drop or redirect traffic based on IP/rate; at app layer, shed based on request priority or user tier. Choose based on cost of rejection and business impact.
Circuit breakers and backpressure: circuit breakers trip after failure threshold, preventing cascading failures; backpressure signals upstream to slow down (e.g., HTTP 429 with Retry-After).
Metrics: track p50/p95/p99 latency, error rates (4xx/5xx), queue depths, rate limit hits, circuit breaker state, and saturation (CPU, memory, connections).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.