## Scenario Design a platform that publishes model-weight artifacts ranging from tens to hundreds of gigabytes to a large fleet of GPU workers. A worker must receive the complete, correct version before serving inference traffic. Operators need to understand nodes with unknown capacity, slow or stuck transfers, checksum failures, and deployments that make no progress. ### Constraints & Assumptions - Artifacts are immutable once published and are addressed by a version plus a cryptographic manifest. - Thousands of workers may update at once; the origin cannot serve a full copy independently to every worker. - GPU hosts have finite local storage and may already cache older versions. - A process can crash after downloading some shards or after loading weights but before becoming routable. - Rollout and rollback must avoid routing requests to partially loaded or wrong-version instances. ### Clarifying Questions to Ask - What rollout time, regional bandwidth, artifact size, and worker count should be assumed? - Are weights sharded by tensor parallelism, and does each worker need all shards or a role-specific subset? - May peer-to-peer distribution be used, or only approved object and cache tiers? - How many versions must coexist for rollback? - What health signal proves that a model is loaded and can execute a representative inference? ```hint Treat publication and readiness as different state machines An artifact can be valid in storage while a worker is still reserving space, downloading, verifying, loading, warming, or failing. ``` ```hint Route by attested version The data plane should select only endpoints whose control-plane record and local readiness check agree on the exact manifest digest. ``` ### What a Strong Answer Covers - Immutable chunked artifacts, manifests, hashes, signatures, and atomic promotion of a release version. - Hierarchical distribution through object storage, regional caches, or controlled peer assistance with bounded origin load. - Resumable parallel download, disk reservation, checksum verification, atomic local activation, and garbage collection. - Explicit worker and rollout state machines with leases, deadlines, retry budgets, and stuck-progress detection. - Capacity discovery and admission that distinguish unknown, reserved, available, and insufficient nodes. - Readiness and routing bound to exact version, load completion, warmup, and a functional probe. - Staged rollout, rollback, mixed-version policy, observability, and blast-radius controls. ### Follow-up Questions 1. How do you prevent a thundering herd when a new version is promoted? 2. What evidence distinguishes slow network transfer from slow disk or model loading? 3. How does a worker resume safely after a crash midway through one shard? 4. Can old and new model versions serve simultaneously, and how is request affinity handled? 5. What happens when a node's reported free capacity is missing or stale?