Design a Scalable Audio AI Processing Platform

# Design a Scalable Audio AI Processing Platform Design the backend for an audio AI processing platform. Clients upload audio and request a multi-stage workflow such as validation, decoding, segmentation, model inference, post-processing, and quality checks. Jobs may be interactive or submitted in large batches. Explain how the system scales, orchestrates durable workflows, uses GPU capacity efficiently, and decides whether an output passes quality checks or requires retry, fallback, or human review. ### Constraints & Assumptions - Audio objects may be much larger than ordinary API payloads and must not travel repeatedly through the control plane. - Individual workflow steps may fail, time out, or be retried, and a job must not silently skip a required stage. - GPU model families may have different memory needs and batching behavior. - Interactive work values latency, while batch work values throughput and cost. - Exact audio duration, arrival rate, model set, quality targets, retention, and regional requirements should be clarified. ### Clarifying Questions to Ask - What does the platform produce: transcripts, classifications, transformed audio, synthesized speech, or several artifact types? - Which workflows require interactive latency, and which may complete asynchronously? - Can long audio be split independently, and what context must overlap between segments? - Which quality failures can be retried automatically, and which need a different model or human review? - Are customers allowed to choose a model version, or must the platform pin and audit versions centrally? ### What a Strong Answer Covers - Direct object-storage upload, immutable input metadata, validation, and a control-plane API that returns a durable job ID. - A persisted workflow state machine with explicit dependencies, idempotent step attempts, retry policy, cancellation, and recovery after orchestrator failure. - Separate scheduling for interactive and batch queues, with fairness, admission control, and backpressure. - GPU-aware placement by model, accelerator type, memory, and deadline; safe dynamic batching and model warm-pool decisions. - Segmenting and recombining long audio without duplicate or missing ranges, including overlap and ordering rules. - Quality gates using stage-specific signals, calibrated thresholds, fallback paths, and versioned human-reviewed evaluation sets. - Artifact lineage from input through model, configuration, step attempts, quality results, and final output. - Failure isolation, cost and capacity measurements, privacy controls, observability, and degraded behavior when GPU capacity is unavailable. ### Follow-up Questions 1. How do you keep a large batch customer from starving low-latency interactive jobs? 2. A worker completes GPU inference but crashes before acknowledging the step. How does a retry avoid corrupting the final result? 3. When does dynamic batching improve GPU use, and when can it violate an interactive deadline? 4. How do you quality-check a long file when each segment looks valid but boundaries create duplicated or missing words? 5. How would you compare a new model version with the current version before routing all production audio to it?