Start by clarifying requirements and constraints, then propose a consistent hashing scheme for shard selection and a log-structured file format per shard. Walk through the write path, shutdown flush with fsync, and startup recovery that rebuilds the in-memory index by scanning files.
Pro tip: Emphasize durability guarantees: use fsync on file and directory, and consider a write-ahead log or checksums to handle partial writes. Also discuss trade-offs between read/write amplification and recovery time.
Ask about expected data size, read/write ratio, latency requirements, and consistency needs. Confirm that files are fixed-size and serialization helpers are provided.
Propose a sharding strategy, e.g., consistent hashing or modulo on key hash, to map keys to shards. Discuss how to handle shard growth and rebalancing if needed.
Describe a log-structured format: append-only records with key, value, timestamp, and checksum. Mention compaction or garbage collection for space reclamation.
On shutdown, flush all in-memory buffers to disk, fsync files and directory, and optionally write a manifest. Discuss atomicity and crash consistency.
On startup, scan each shard file, validate checksums, and rebuild the in-memory index (e.g., hash map) by replaying records. Handle partial writes and corruption.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.