Verkada Inc.·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

Verkada system design round, one meaty question about real-time device telemetry at scale. The whole session was basically a single deep-dive and they just kept pulling on threads until time ran out.

Questions Asked (1)

Design a system that handles 1 million devices each sending a heartbeat every minute with one of 10 possible status values. The system needs to compute, in real time, the count of devices per status over the previous minute window. Walk through ingestion, time-bucketing, partitioning, aggregation approach, state storage with TTL, fault tolerance, late and duplicate events, and how to serve the per-status counts with low latency.

System DesignTechnical Trade-offsData Modeling

Author's notes

This one sprawled.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale (1M devices/min ≈ 16.7K events/sec), then propose a streaming pipeline with time-bucketed windows (e.g., 1-minute tumbling windows) and a distributed aggregation layer. Emphasize partitioning by device ID for even load, stateful processing with TTL for window state, and a serving layer that exposes per-status counts via a low-latency store like Redis.

Pro tip: Mention that you can pre-aggregate counts per status within each partition before merging, reducing data shuffling and enabling near real-time updates; also discuss using a time-series database or Redis sorted sets for efficient windowed counts.

1. Clarify Requirements and Scale

Confirm the event rate (1M/min ≈ 16.7K/sec), latency requirements (real-time, likely sub-second), and whether exactly-once semantics are needed. Discuss the 10 status values and the need for per-minute counts.

2. Design Ingestion and Partitioning

Use a scalable message queue (e.g., Kafka) with partitions keyed by device ID to ensure ordered processing per device and even distribution. Consider batching to reduce overhead.

3. Implement Time-Bucketing and Aggregation

Apply tumbling windows of 1 minute (or sliding windows if needed) using a stream processor (e.g., Flink, Spark Streaming). Aggregate counts per status within each window, leveraging local pre-aggregation before global merge.

4. Manage State and Fault Tolerance

Store window state in a distributed store (e.g., RocksDB with Flink) with TTL to expire old windows. Use checkpointing and exactly-once semantics to handle failures; ensure idempotent updates for duplicates.

5. Serve Low-Latency Counts

Write aggregated counts to a fast serving layer (e.g., Redis) keyed by window timestamp and status. Expose an API that reads the latest window counts, with caching and possibly pre-computed results for the previous minute.

Key Points to Mention

Use of tumbling windows for 1-minute intervals and handling of late/out-of-order events with watermarks or allowed lateness.
Partitioning strategy: key by device ID to ensure per-device ordering and even load distribution across partitions.
State management with TTL: store window state in a distributed store with time-to-live to automatically purge old data.
Fault tolerance: checkpointing, exactly-once processing, and idempotent writes to handle failures and duplicates.
Low-latency serving: pre-aggregated counts in Redis or similar, with an API that returns the latest window counts quickly.
Scalability: horizontal scaling of ingestion and processing layers to handle 1M devices and potential growth.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.