rippling·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

Rippling system design round, one big question about building an event ingestion pipeline that serves both a real-time dashboard and a batch processing layer. Pretty open-ended, which I wasn't fully prepared for.

Questions Asked (1)

Design a system that collects user behavior events (installs, clicks, etc.) and supports two outputs: a real-time metrics dashboard with low latency, and a raw event store for large-scale offline batch processing. Walk through your choices for collection, transport, stream processing, and storage.

System DesignTechnical Trade-offsProduct Analytics & Metrics

Author's notes

The dual-output requirement is what made this interesting.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements: event types, scale (events per second), latency targets for real-time metrics, and batch processing needs. Then propose a high-level architecture with distinct paths for real-time and batch, and dive into each component (collection, transport, stream processing, storage) discussing trade-offs and choices. Finally, address scalability, fault tolerance, and data consistency.

Pro tip: Emphasize the importance of a unified event schema and idempotent processing to avoid data duplication or loss, and mention how you would handle late-arriving events in both paths.

1. Clarify Requirements and Constraints

Ask about expected event volume, latency requirements for real-time dashboard (e.g., sub-second vs. seconds), batch processing frequency, data retention, and consistency needs. This shapes technology choices.

2. Design Collection and Transport Layer

Propose a scalable ingestion layer (e.g., HTTP endpoints, SDKs) that buffers and forwards events to a durable message queue like Kafka. Discuss partitioning, replication, and backpressure handling.

3. Design Real-Time Stream Processing Path

Use a stream processor (e.g., Flink, Spark Streaming, Kafka Streams) to compute metrics in near real-time, writing to a fast store (e.g., Redis, Druid, ClickHouse) for low-latency dashboard queries.

4. Design Batch Storage and Processing Path

Persist raw events to a scalable, cost-effective store (e.g., S3, HDFS) in a columnar format (Parquet) for offline batch processing with frameworks like Spark or Hive.

5. Address Scalability, Fault Tolerance, and Consistency

Discuss how to scale each component, handle failures (e.g., Kafka replication, checkpointing in stream processors), and ensure exactly-once or at-least-once semantics with idempotent writes.

Key Points to Mention

Use of Kafka as a durable, scalable buffer between collection and processing
Lambda architecture vs. Kappa architecture trade-offs for real-time and batch
Choice of stream processing framework (e.g., Flink for exactly-once, low latency)
Storage options: fast OLAP database for real-time (e.g., Druid, ClickHouse) and object storage for batch (e.g., S3 with Parquet)
Handling late/out-of-order events with watermarks or windowing
Monitoring, alerting, and data quality checks in the pipeline

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.