# Design a Fault-Tolerant Video Generation Platform ### Prompt Design an asynchronous video-generation platform. A user submits generation parameters, checks progress, and retrieves a finished video. The underlying generation may require several expensive distributed stages, but model internals are out of scope. Focus on scheduling, state management, artifact handling, and failure models. For each major component, explain what happens if it becomes slow, unavailable, duplicates work, or loses a response. **Candidate hint:** At-least-once delivery is manageable only if attempt identity and the right to publish a result are explicit. ### Constraints & Assumptions - Jobs may run much longer than an HTTP request and can exceed a worker lease. - Intermediate and final artifacts are large. - Workers can crash or become unreachable after completing a stage. - Clients may retry submissions and status requests. - State assumptions about stage dependencies, checkpointing, cancellation, and output retention. ### Clarifying Questions to Ask - Are stages restartable and deterministic for a fixed seed and input? - What progress information must be accurate versus approximate? - Is partial output useful, or is only a complete video publishable? - What latency, cost, and durability targets should drive trade-offs? ### What a Strong Answer Covers - Durable request and attempt state with idempotent public APIs - Dependency-aware scheduling, leases, retries, and bounded duplicate execution - Correct publication of large, immutable artifacts - Backpressure, fairness, and protection of scarce compute - Failure analysis for API, metadata store, queue, scheduler, worker, and object storage - Recovery, observability, cleanup, and security controls ### Follow-up Questions 1. How do you handle a worker that completes after the scheduler has reassigned its attempt? 2. What state must be transactionally coupled to queue publication? 3. How would you resume a multi-stage job after losing the scheduler entirely? 4. How can the system shed load without losing already accepted work?