Amazon·Software Engineer·Onsite - System Design / Architecture·Senior

Senior

Jun 2026

Summary

Amazon system design round, low-level design focus. The whole session was basically one big question about building a job scheduler from scratch, and they wanted a lot of detail across a lot of dimensions.

Questions Asked (1)

Design a job scheduling system that can execute a job X seconds in the future. Cover the API design, internal data structures, persistence, worker model, restart/clock drift handling, delivery semantics, concurrency, time complexity, and how you'd test it.

System DesignTechnical Trade-offsAlgorithms & Data Structures

Author's notes

This one sprawled in every direction.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements (scale, latency, durability, delivery semantics) and then design a system that stores jobs in a persistent store, uses a timing mechanism to trigger execution, and distributes work across workers. Focus on trade-offs between accuracy, scalability, and complexity, and address failure modes like restarts and clock drift.

Pro tip: Emphasize idempotency and at-least-once delivery with deduplication, as Amazon values reliability and customer trust. Also, discuss how you'd monitor and alert on scheduling delays and failures.

1. Clarify Requirements and Scope

Ask questions to understand scale (jobs per second, max delay), latency requirements, durability needs, and delivery semantics (at-least-once vs exactly-once). This shapes the entire design.

2. High-Level Architecture

Propose a distributed system with a persistent job store (e.g., database), a scheduler service that polls or uses timers, and a pool of workers that execute jobs. Consider using a message queue for decoupling.

3. Data Model and Persistence

Design a schema for jobs with fields like job_id, payload, execute_at, status, and retry_count. Use a database with indexing on execute_at for efficient querying, and ensure durability via replication.

4. Scheduling and Execution

Explain how to efficiently find due jobs: use a priority queue (min-heap) in memory for fast access, backed by the database. Discuss worker model: pull-based vs push-based, and how to handle concurrency with locking or optimistic concurrency control.

5. Failure Handling and Testing

Address restart recovery by reloading future jobs from the database, clock drift by using NTP and periodic resync, and delivery semantics with idempotent workers and deduplication. Describe testing strategies: unit tests for timing logic, integration tests with simulated failures, and load tests.

Key Points to Mention

Use of a min-heap or timing wheel for efficient in-memory scheduling, with time complexity O(log n) for insertion and O(1) for peeking the next job.
Persistence layer with a database (e.g., DynamoDB, RDS) and indexing on execute_at to efficiently query due jobs; consider sharding for scale.
Worker model: multiple workers pulling jobs from a queue or database, with locking to prevent duplicate execution; use of visibility timeouts for at-least-once delivery.
Handling clock drift: synchronize clocks with NTP, use a central time service, and design for idempotent job execution to tolerate duplicate runs.
Restart recovery: on startup, workers load pending jobs from the database and reschedule; use a leader election or distributed lock to avoid duplicate scheduling.
Testing: unit tests for scheduling logic, integration tests with fault injection (e.g., worker crashes, clock skew), and load testing to ensure scalability.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.