Google·Software Engineer·Onsite - System Design / Architecture·Senior

Senior

Jun 2026

Summary

System design round at Google for a software engineering role. The whole session was one big question about building a replay system on top of an existing pub/sub setup, and it went pretty deep across like six different dimensions. Dense but interesting.

Questions Asked (1)

Given an existing pub/sub messaging system, design a replay system that lets consumers reprocess historical messages from a specific point in time or offset, without affecting live traffic. Cover durable message retention and indexing, replay APIs, isolation from live consumers, ordering and delivery semantics, schema evolution, and crash recovery.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

This was basically six questions stapled together and I did not pace myself well.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale, then propose a design that separates storage (durable log with indexing) from replay serving (dedicated replay consumers or a replay service). Walk through the core components: retention, indexing, replay APIs, isolation, ordering, schema evolution, and crash recovery, while discussing trade-offs at each step.

Pro tip: Emphasize that replay should be a first-class feature with its own resource quotas and backpressure, not an afterthought; this prevents replay from impacting live traffic and shows production maturity.

1. Clarify Requirements and Scale

Ask about message volume, retention period, replay frequency, latency requirements, and consistency needs to scope the design appropriately.

2. Design Durable Storage and Indexing

Propose a partitioned, append-only log with time-based and offset-based indexes to enable efficient seeking to a specific point in time or offset.

3. Define Replay APIs and Isolation

Specify APIs for initiating, monitoring, and canceling replays, and ensure isolation by using separate consumer groups or dedicated replay clusters with resource quotas.

4. Address Ordering, Delivery, and Schema Evolution

Guarantee per-partition ordering, at-least-once delivery, and handle schema evolution by storing schema versions and providing compatibility checks.

5. Plan for Crash Recovery and Monitoring

Describe checkpointing replay progress, idempotent processing, and monitoring replay lag and resource usage to recover from failures without data loss.

Key Points to Mention

Durable retention with tiered storage (e.g., hot/warm/cold) to balance cost and replay latency.
Indexing strategies: time-based indexes, offset indexes, and possibly secondary indexes for message attributes.
Replay API design: start/stop replay, query replay status, and specify start point (timestamp/offset).
Isolation techniques: separate consumer groups, dedicated replay clusters, and resource quotas to prevent live traffic impact.
Ordering and delivery semantics: per-partition ordering, at-least-once vs exactly-once, and idempotent consumers.
Schema evolution: schema registry, versioning, and backward/forward compatibility checks during replay.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.