Verkada Inc.·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Apr 2026

Summary

System design round at Verkada for a software engineering role, one big question about monitoring cameras at scale. The whole session was basically a deep dive into a single problem and they pushed hard on every layer of it.

Questions Asked (1)

Design a system that monitors 10 million cameras, each sending a heartbeat signal once per minute. The system needs to report healthy vs unhealthy device counts in near real time, handle ingestion at scale, and correctly mark a device as unhealthy if it misses heartbeats within a sliding time window.

System DesignTechnical Trade-offsData Modeling

Author's notes

This one took me a second to size up.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale (10M devices, 1 heartbeat/min = ~167K QPS). Then propose a horizontally scalable ingestion pipeline (e.g., Kafka) and a stateful processing layer (e.g., Flink) that maintains per-device last-seen timestamps and emits health status changes. Finally, discuss storage for real-time counts and trade-offs around windowing, fault tolerance, and cost.

Pro tip: Emphasize that you would avoid per-heartbeat database writes by using an in-memory state store with periodic checkpoints, and that you would shard devices to distribute load evenly. This shows you understand both scalability and operational cost.

1. Clarify requirements and constraints

Ask about expected latency for health status updates, acceptable false positives/negatives, and whether historical data is needed. Confirm the scale: 10M devices, 1 heartbeat/min = ~167K writes/sec.

2. Design ingestion layer

Propose a distributed message queue (e.g., Kafka) to absorb the heartbeat stream, with partitioning by device ID to ensure ordered processing per device. Mention the need for backpressure and replication.

3. Design processing and state management

Use a stream processor (e.g., Flink) with keyed state to track last heartbeat time per device. Implement a sliding window or timer to detect missed heartbeats (e.g., if no heartbeat in 2 minutes, mark unhealthy). Emit state change events.

4. Design storage and query layer

Store health status in a fast key-value store (e.g., Redis) for real-time counts, and optionally in a time-series DB for analytics. Use a separate service to aggregate counts and expose an API for near real-time dashboards.

5. Address trade-offs and failure handling

Discuss trade-offs: exactly-once vs at-least-once processing, window size vs detection latency, and cost of state storage. Explain how to handle node failures, rebalancing, and recovery from checkpoints.

Key Points to Mention

Partitioning by device ID to ensure ordered processing and scalability
Using stream processing with keyed state and timers for sliding window detection
Avoiding per-heartbeat database writes by using in-memory state with periodic checkpoints
Separating ingestion, processing, and query layers for scalability and fault tolerance
Trade-offs between detection latency and false positives (e.g., window size, grace period)
Handling device clock skew and out-of-order heartbeats

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.