← Snowflake Interview Insights
Start by clarifying requirements (file sizes, access patterns, consistency, scale) and then design a two-layer architecture: a metadata service and a blob store. Focus on content-addressable storage using cryptographic hashes for deduplication, and discuss trade-offs like hash collisions, garbage collection, and performance.
Pro tip: Mention that deduplication can be done at different granularities (whole-file vs. chunk-level) and that Snowflake's data cloud likely benefits from chunk-level dedup for large files; also highlight the importance of a reference counting mechanism to safely delete objects.
Ask about expected file sizes, upload/download frequency, consistency needs, and scale (number of files, total storage). This ensures the design meets the actual use case.
Propose a separation of concerns: a metadata service (tracking file info, ownership, references) and a blob storage layer (storing actual data). Use a hash-based addressing scheme for deduplication.
Explain content-addressable storage: compute a cryptographic hash (e.g., SHA-256) of the file content. If the hash exists, increment a reference count and return the existing object; otherwise, store the new object.
Detail the steps: client uploads file, server hashes it, checks metadata store for existing hash, stores if new, updates metadata with reference. Download: client requests file by ID, metadata service returns hash, client fetches from blob store.
Discuss hash collisions (use strong hash, verify content), garbage collection (reference counting, mark-and-sweep), performance (caching, CDN), and consistency (eventual vs. strong).
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
The reference counting angle is where things got messy for me.
Start by clarifying the system's requirements: how files are shared, what consistency guarantees are needed, and the expected scale. Then propose a reference counting mechanism to track shared content, with a two-phase deletion process (mark and sweep) to handle concurrent access safely. Finally, discuss trade-offs between eager and lazy garbage collection, and how to handle edge cases like partial failures and orphaned data.
Pro tip: Emphasize that deletion should be logical (tombstoning) rather than physical to avoid race conditions, and that garbage collection should be asynchronous and idempotent. Mention that Snowflake's architecture likely uses a similar approach with metadata management and background processes.
Ask about the sharing model (e.g., deduplication, copy-on-write), consistency requirements, and scale. Understand if users can delete while others are reading, and what latency is acceptable for cleanup.
Propose maintaining a reference count per unique file content. Increment when a user references it, decrement on deletion. Only physically delete when count reaches zero.
Use a two-phase approach: mark for deletion (logical delete) and then garbage collect asynchronously. Ensure atomicity of reference count updates to avoid races.
Address race conditions (e.g., simultaneous delete and read) with locking or optimistic concurrency. Plan for partial failures: if a user crashes, reference counts may leak; use timeouts or leases.
Compare eager vs. lazy GC, synchronous vs. asynchronous cleanup, and the impact on performance and storage. Mention potential optimizations like batching or background compaction.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
I talked about chunked uploads with a manifest commit at the end, so a partial upload never becomes visible.
Start by clarifying the upload model (e.g., chunked, multipart, or streaming) and the consistency guarantees required. Then systematically discuss correctness risks such as partial visibility, duplicate data, and ordering issues, and propose concrete solutions like idempotent writes, transactional commits, and checksum validation. Finally, tie your answer to Snowflake's architecture, emphasizing how its cloud-native design handles these concerns.
Pro tip: Mention that Snowflake's internal stages and COPY command already provide atomicity and idempotency via file checksums and metadata, so you'd leverage those rather than reinventing the wheel. This shows you understand the product and avoid over-engineering.
Ask whether uploads are chunked, multipart, or streaming, and what consistency level is needed (e.g., atomic visibility, exactly-once). This scopes the problem and shows you avoid assumptions.
Enumerate issues like partial uploads leaving incomplete data, concurrent uploads causing duplicates or lost updates, and out-of-order chunks leading to corruption. Also consider failure scenarios (network, client crash).
Suggest solutions such as idempotent writes with unique upload IDs, transactional commit of all chunks, checksums for integrity, and versioning or optimistic concurrency control for concurrent updates.
Discuss how to handle concurrent uploads to the same object: use locking, conditional writes, or append-only logs with deduplication. Ensure readers see a consistent snapshot (e.g., via MVCC or staging areas).
Explain how Snowflake's stages, COPY command, and metadata services provide atomicity and idempotency. Acknowledge trade-offs between consistency, latency, and complexity, and justify your choices.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.