Google·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

System design round at Google for a software engineering role. The whole thing centered on one big question about notification delivery guarantees, and it went deeper than I expected pretty fast.

Questions Asked (1)

Design a notification system that guarantees at-least-once delivery. Cover producer and consumer interfaces, deduplication keys, retry with exponential backoff, out-of-order message handling, and idempotency on the user-facing side.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

I started with the queue and producer interface which felt natural, but then they kept pulling the thread.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements (scale, latency, ordering) and then design the system in layers: producer API, durable queue, consumer with retries, and idempotent user-facing handler. Emphasize trade-offs between at-least-once and exactly-once, and how deduplication and idempotency mitigate duplicates.

Pro tip: Proactively discuss how you'd monitor and alert on duplicate rates and retry queues, and how you'd handle poison messages to avoid infinite retries.

1. Clarify Requirements and Scope

Ask about expected throughput, latency, ordering guarantees, and failure modes. Define what 'at-least-once' means for the system and how duplicates will be handled downstream.

2. Design Producer and Consumer Interfaces

Define APIs for producing messages (e.g., HTTP POST with payload and dedup key) and consuming (e.g., pull-based with ack). Include metadata like message ID, timestamp, and retry count.

3. Implement Deduplication and Retry Logic

Use a deduplication key (e.g., UUID) stored in a fast lookup store (e.g., Redis) to filter duplicates. Implement exponential backoff with jitter for retries, and a dead-letter queue for poison messages.

4. Handle Out-of-Order and Idempotency

Use sequence numbers or timestamps to detect and reorder messages if needed. Ensure user-facing operations are idempotent by using idempotency keys or upsert semantics.

5. Discuss Trade-offs and Monitoring

Compare at-least-once vs exactly-once, and explain how you'd monitor duplicate rates, retry queues, and system health. Mention how to scale components horizontally.

Key Points to Mention

Deduplication keys: unique message IDs and a dedup store with TTL to prevent duplicates.
Retry with exponential backoff and jitter: avoid thundering herd, set max retries, and use a dead-letter queue.
Out-of-order handling: sequence numbers or timestamps, and consumer-side reordering buffers if needed.
Idempotency on user-facing side: idempotency keys, upsert operations, or conditional writes to ensure repeated processing doesn't cause side effects.
At-least-once vs exactly-once: acknowledge that exactly-once is harder and often requires idempotency; at-least-once with dedup is practical.
Monitoring and alerting: track duplicate rates, retry counts, DLQ size, and latency to ensure system health.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.