How to Design a Distributed Job Scheduler in a System Design Interview

"Design a cron service for thousands of machines" tests whether you can reason about time, retries and exactly-once execution. Here is a compact answer outline.

1. Requirements

  • Users register one-off or recurring jobs (cron expressions) with a payload and a target.

  • Jobs should start within a few seconds of their scheduled time.

  • A job must not be silently lost, and should not run twice when that matters.

  • Scale: millions of jobs, tens of thousands firing per minute at peaks such as the top of the hour.

2. Data model

A jobs table stores the definition (id, owner, schedule, payload, retry policy). An executions table stores each run with a status: scheduled, running, succeeded, failed. The key index is on next_run_time.

3. Scheduling loop

Scheduler workers poll for jobs whose next_run_time is due, partitioned by job id hash so each worker owns a slice. A worker claims a job with a conditional update (or a lease with an expiry), pushes an execution message to a queue, and computes the next run time. Using time buckets (for example one bucket per minute) keeps the polling query cheap.

4. Execution

Executor workers consume the queue, run the job, and heartbeat while running. If heartbeats stop, the lease expires and another worker retries. Retries use exponential backoff with a max attempt count, then the run goes to a dead-letter queue for alerting.

5. Exactly-once is really at-least-once plus idempotency

Explain that a network partition can always cause a duplicate run, so jobs receive an execution id and downstream systems deduplicate on it. This is the point interviewers most often probe.

6. Hot spots and fairness

Everyone schedules jobs at 00:00, so add jitter for jobs that allow it, rate limit per tenant, and scale executors from queue depth. Mention monitoring: schedule lag (actual start minus planned start) is the main health metric.

Practice tip

Draw the claim-execute-heartbeat cycle and walk through one failure. For rehearsal I like using an AI assistant for technical interviews that throws follow-up questions at me (disclosure: I work on Linkjob AI).