I started with ingestion and got a bit stuck on the time-decay part longer than I should have.
Start by clarifying requirements and constraints, then propose a high-level architecture that separates ingestion, aggregation, ranking, and serving. Walk through the data flow, emphasizing how to handle time-decay and top-K efficiently, and discuss trade-offs between real-time and batch processing.
Pro tip: Mention that ranking signals should be aggregated in a way that supports incremental updates and that the time window is configurable, so the system must handle both sliding and tumbling windows. Also, highlight the importance of idempotency and exactly-once processing in event ingestion to avoid skewed rankings.
Ask about expected QPS, number of posts, time window granularity, and latency requirements. Define what 'hottest' means (e.g., upvotes, comments, shares) and how configurable the window is.
Propose a scalable ingestion layer (e.g., Kafka) to collect user interactions. Ensure events are partitioned by post ID for ordered processing and discuss exactly-once semantics.
Use a stream processor (e.g., Flink) to compute a score per post using a decay function (e.g., exponential or linear). Maintain a sliding window of events and update scores incrementally.
Employ a distributed top-K algorithm (e.g., using a heap or count-min sketch with heap) to track the highest-scoring posts. Periodically merge results from partitions and handle late data.
Store the top-K results in a low-latency store (e.g., Redis) for fast retrieval. Discuss batch recomputation for correction and the trade-offs between real-time and batch processing.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.