The core challenge isn't the percentile math, it's making both methods thread-safe without killing performance.
Start by clarifying requirements: expected throughput, latency distribution, window size, and precision. Then propose a design using a ring buffer of time buckets, each with a thread-safe histogram (e.g., HdrHistogram) for efficient percentile calculation. Discuss trade-offs between accuracy, memory, and performance, and how to handle concurrency with minimal contention.
Pro tip: Emphasize that Netflix cares about real-time performance at scale, so mention using lock-free data structures or fine-grained locking to avoid bottlenecks, and consider using a histogram with configurable precision to balance accuracy and memory.
Ask about expected write throughput, read frequency, window size granularity, and acceptable error margin for percentile. This shows you think about practical constraints.
Propose a ring buffer of time buckets (e.g., 1-second buckets) covering the window. Each bucket contains a histogram (e.g., HdrHistogram) to store latencies. This allows efficient sliding window and percentile calculation.
Use thread-local histograms or striped locks to reduce contention on writes. For reads, aggregate histograms from buckets within the window, possibly using a read-write lock or snapshot to ensure consistency.
For getP99Latency, merge histograms from relevant buckets and compute the 99th percentile. Discuss using a histogram's percentile method or maintaining a sorted structure if exact percentile is needed.
Compare exact vs approximate percentiles, memory usage, and latency. Mention potential optimizations like pre-aggregation, sampling, or using a t-digest for streaming percentiles.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.