Datadog·Software Engineer·Onsite - System Design / Architecture·Senior

Senior

May 2026

Summary

Datadog system design round, one big question about building a log ingestion and query system from scratch. Pretty intense for a single question but they really dug into the details across the full hour.

Questions Asked (1)

Design a data structure that records log entries and supports efficient querying. Each log has a timestamp in milliseconds, a severity level, a service ID, and a message. You need to support: appending a log, counting logs in a time range with optional filters, and fetching logs in a time range with filters, ordering, and pagination. Walk through your data layout, indexing strategy, time-range scan approach, and the complexity tradeoffs between write amplification and query latency.

System DesignAlgorithms & Data StructuresTechnical Trade-offs

Author's notes

I started with a sorted structure keyed on timestamp, which felt right, but then they pushed on what happens when you add severity and serviceId filters on top of the range scan and I kind of fumbled.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale, then propose a time-partitioned storage layout with secondary indexes on severity and service ID. Explain how to combine time-range scans with filter intersection, and discuss tradeoffs between write amplification and query latency, including pagination and ordering strategies.

Pro tip: Emphasize that real-world log systems often use a hybrid approach: in-memory buffers for recent writes and immutable on-disk segments for older data, which balances write throughput and query performance. Also, mention that pagination should be based on a stable sort key (timestamp + unique ID) to avoid duplicates or missing entries.

1. Clarify requirements and scale

Ask about write volume, query patterns, latency SLAs, and retention. This determines whether to optimize for writes or reads.

2. Design data layout and indexing

Propose time-partitioned storage (e.g., hourly/daily segments) with secondary indexes on severity and service ID. Consider inverted indexes or bitmap indexes for filters.

3. Implement time-range scan and filter intersection

For a query, identify relevant time partitions, then use indexes to quickly find matching logs. Intersect posting lists for multiple filters.

4. Handle ordering and pagination

Sort results by timestamp (and a tiebreaker like log ID). Use cursor-based pagination with the last seen timestamp+ID to avoid deep offsets.

5. Discuss complexity and tradeoffs

Analyze write amplification (e.g., updating indexes) vs query latency. Mention LSM-trees, columnar storage, or hybrid approaches.

Key Points to Mention

Time-based partitioning (e.g., hourly segments) to limit scan scope
Secondary indexes on severity and service ID, using inverted indexes or bitmaps
Write amplification from index updates and how to mitigate (e.g., batch writes, LSM-trees)
Query latency optimization via filter intersection and early termination
Cursor-based pagination with stable sort key (timestamp + unique ID)
Tradeoffs: in-memory vs on-disk, columnar vs row-based storage, and retention policies

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.