# Design a Video Generation Service with GPU Fleet Management Design a service that accepts text or media prompts and asynchronously generates videos. Focus on scheduling and operating the GPU worker fleet. The detailed numbers and product rules below are practice assumptions. ### Constraints & Assumptions - Jobs vary by model, resolution, duration, and expected GPU memory. - Generation takes minutes, can be checkpointed at selected stages, and may fail. - Users need job status, cancellation, result retrieval, and bounded retry behavior. - The fleet contains multiple GPU types across regions and can autoscale. - Safety checks run before and after generation. ### Clarifying Questions to Ask - Which latency, throughput, quality, and availability targets matter? - Can a job move between GPU types or resume on another node? - How are priorities, quotas, and paid tiers defined? - What input and output retention or privacy requirements apply? ### Part 1: Job and Control Plane Define APIs, job states, metadata, model and artifact versions, queues, and idempotency. #### Hints - Submission, scheduling, execution, and publication are separate transitions. #### What This Part Should Cover - Durable state machine - Versioned reproducibility - Quotas and cancellation semantics ### Part 2: GPU Scheduling and Node Lifecycle Explain placement, admission control, batching, preemption, leases, heartbeats, autoscaling, and handling a lost or unhealthy GPU. #### Hints - GPU memory, model residency, and queue delay can pull placement in different directions. #### What This Part Should Cover - Resource-aware scheduler - Fairness and utilization trade-offs - Fenced ownership and recovery ### Part 3: Reliability, Safety, and Cost Cover checkpointing, retries, artifact publication, moderation, regional failures, cost controls, and observability. #### Hints - Retrying an expensive job can be correct and still be operationally harmful. #### What This Part Should Cover - Idempotent output path - Bounded failure handling - Quality, safety, and fleet signals ### What a Strong Answer Covers - A durable asynchronous job architecture - GPU-aware placement and explicit fairness policies - Safe recovery, checkpoint, and publication semantics - Autoscaling, moderation, cost, and measurable fleet operation ### Follow-up Questions - How would you schedule a new model that fits only one GPU type? - When should the system preempt an in-progress job? - How would you prevent a retry storm after a regional outage? - What changes for interactive preview generation?