# Schedule One-GPU and Eight-GPU Model Jobs Design a scheduler for a fixed pool of eight GPUs. Small-model jobs need exactly one GPU, while large-model jobs need all eight GPUs at the same time. The workload is online and job durations vary. Explain the scheduling policy, admission control, starvation prevention, and behavior when small jobs continuously arrive. ### Constraints & Assumptions - A GPU can run at most one admitted job at a time. - Large jobs are gang scheduled and cannot start with fewer than eight GPUs. - Jobs are not preemptible in the base design. - Each job has arrival time, class, priority, and an estimated duration. - The product needs bounded waiting for both classes. ### Clarifying Questions to Ask - Are large-job deadlines or priorities more important than utilization? - Can small jobs be delayed before a large job arrives? - How accurate are duration estimates? - Is preemption or checkpointing available as an extension? ### Part 1 - Queueing and admission Define queue state, capacity accounting, and when new small jobs should stop entering GPUs so a queued large job can start. #### What This Part Should Cover - Separate class queues and global resource state - Reservation or drain policy for gang jobs - Backpressure when waiting bounds cannot be met - Race-free all-eight allocation ### Part 2 - Fairness and utilization Compare strict priority, reservation windows, aging, and workload-aware policies. State how starvation is prevented. #### What This Part Should Cover - Trade-off between high utilization and large-job latency - Aging or bounded drain initiation - Small-job throughput during unreserved periods - Robustness to inaccurate duration estimates ### Part 3 - Failures and measurement Discuss worker failure, job cancellation, state recovery, and the metrics or simulation used to tune the policy. #### What This Part Should Cover - Transactional gang allocation - Leases and reconciliation - Queue-time percentiles by class - Utilization and starvation evidence ```hint A large job needs a drain point Without eventually refusing new small work, continuous one-GPU arrivals can keep at least one device occupied forever and starve every eight-GPU job. ``` ### What a Strong Answer Covers - Atomic gang allocation and accurate resource state - An explicit drain/reservation policy that bounds large-job starvation - A fair treatment of small jobs and honest utilization trade-offs - Recovery, admission control, and measurable tuning criteria ### Follow-up Questions 1. How would checkpointable small jobs change the policy? 2. What if a large job uses eight GPUs but has much lower business priority? 3. How would the design generalize to several nodes with topology-sensitive GPU links?