Design Reliable Upload and Download for Very Large Files

## Prompt Design a service that lets clients upload and download files too large to send reliably in one request. The design should support resumable transfer, integrity verification, concurrent clients, authorization, and cleanup of abandoned uploads. ### Constraints & Assumptions - File bytes are stored in object storage; metadata is stored separately. - A client may disconnect after any chunk and resume from another process. - Downloads should support byte ranges and must never expose an uncommitted file. - Retrying a chunk or completion request must be idempotent. ### Clarifying Questions to Ask - What maximum file size, chunk size range, and completion-latency target are expected? - May clients upload chunks out of order, and are file versions immutable after commit? - Which integrity hash is required for chunks and for the complete object? ```hint Commit metadata last Keep an upload invisible until every required chunk and the final digest are verified, then publish one committed metadata version. ``` ```hint Use stable chunk identities Address a chunk by upload session and part number, and bind retries to an expected checksum. ``` ### What a Strong Answer Covers - An upload-session API, chunk addressing scheme, and atomic completion protocol. - Separation of committed metadata from in-progress state. - Integrity, authorization, retry, and duplicate-chunk behavior. - Range-download flow, caching considerations, and backpressure. - Cleanup, observability, abuse controls, and failure recovery. ### Follow-up Questions 1. How would you deduplicate identical complete files without letting one tenant infer another tenant's data? 2. What changes if clients need end-to-end encryption and the server cannot compute the plaintext digest? 3. How would you migrate an object between storage regions without interrupting downloads?