EY·Data Scientist·Onsite - System Design / Architecture·Senior

Senior

Jul 2026

Summary

EY data scientist interview that was basically one big system design question dressed up as a cloud architecture problem. More depth than I expected for a DS role, felt more like a data engineering panel at times.

Questions Asked (1)

Design a cloud data ingestion and serving architecture that handles both streaming and batch workloads. Walk through your choice of ingestion pattern (CDC, event sourcing, or micro-batch), how you'd partition and compact data, how you'd guarantee idempotency and handle schema changes like adding optional fields, how you'd tokenize PII, and what cost controls you'd put in place. Also describe an incident scenario you'd use in chaos testing.

System DesignTechnical Trade-offsData Modeling

Author's notes

This was basically a 45-minute conversation crammed into one question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements (latency, volume, compliance) and then propose a hybrid architecture that uses CDC for real-time ingestion and batch for historical loads, with a unified storage layer. Walk through partitioning, compaction, idempotency, schema evolution, PII tokenization, and cost controls, and finish with a concrete chaos testing incident.

Pro tip: Emphasize that idempotency and schema evolution are best handled at the storage layer (e.g., using Delta Lake or Iceberg) rather than in application code, and always tie cost controls to business value to show maturity.

1. Clarify Requirements and Constraints

Ask about data volume, velocity, latency SLAs, compliance needs (e.g., GDPR), and budget. This ensures your design is tailored and shows you think before architecting.

2. Choose Ingestion Pattern and Data Flow

Select CDC for real-time changes, batch for historical loads, and optionally event sourcing for auditability. Describe how data flows from sources to a unified storage layer (e.g., data lake) and then to serving layers.

3. Design Storage: Partitioning, Compaction, and Idempotency

Explain partitioning strategy (e.g., by date and key), compaction to merge small files, and idempotency via upserts or deduplication keys. Mention using open table formats like Delta Lake or Iceberg.

4. Address Schema Evolution and PII Tokenization

Describe how to handle schema changes (e.g., adding optional fields) using schema registry and backward-compatible formats. Explain PII tokenization using a tokenization service or format-preserving encryption.

5. Implement Cost Controls and Chaos Testing

Outline cost controls like auto-scaling, tiered storage, and query optimization. Then describe a chaos testing incident, such as simulating a Kafka broker failure or schema registry outage, and how the system recovers.

Key Points to Mention

CDC (Change Data Capture) for streaming and batch ingestion patterns
Partitioning by time and key, and compaction strategies (e.g., bin-packing, Z-order)
Idempotency via upserts, deduplication, and exactly-once semantics
Schema evolution using schema registry and backward-compatible formats (Avro, Protobuf)
PII tokenization techniques (tokenization service, format-preserving encryption)
Cost controls: auto-scaling, tiered storage, query optimization, and monitoring

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.