Design a Durable Cron Job Scheduler

Design a durable cron job scheduler that runs recurring jobs according to a schedule. Explain schedule interpretation, dispatch, worker execution, recovery after failures, and behavior when a scheduled run is missed. ### Constraints & Assumptions - The source names the system without scale or delivery guarantees. State a baseline and distinguish execution attempts from logical scheduled occurrences. - Each schedule has an explicit timezone. Clarify daylight-saving transitions and edits before defining next-run behavior. - Do not promise exactly-once external side effects merely because a job record is unique. ### Clarifying Questions to Ask - Should missed runs be skipped, coalesced, or replayed individually? - Can two occurrences of the same job overlap? - Are job actions idempotent, and how should retries and permanent failures be surfaced? - What happens to already-created occurrences when a schedule is edited or disabled? ```hint Give one occurrence a stable identity A scheduler restart can rediscover the same due time. A durable uniqueness rule should distinguish that logical run from a new execution attempt. ``` ### What a Strong Answer Covers - Durable schedule definitions, timezone-aware next-run calculation, and a misfire policy. - Atomic creation of due occurrences and advancement of scheduling state. - Worker claims, leases, retries, and completion semantics. - High-availability scheduling without duplicate logical occurrences. - Operational visibility, backlog control, and schedule-edit behavior. ### Follow-up Questions - What happens if a worker finishes the external action and crashes before recording success? - How would the system recover after an hour-long scheduler outage without overwhelming workers?