Design a Large-File Analysis Report System

# Design a Large-File Upload and Analysis Report System Design a service where an authenticated user uploads a potentially large file, processing runs asynchronously, and the user later views a generated report. Cover upload APIs, progress and retry, object storage, job states, duplicate-analysis prevention, worker scaling, failure recovery, and authorization. ### Constraints & Assumptions - Files may be too large to proxy through an application server or hold in worker memory. - Upload and analysis can each be retried after interruption. - Analysis may take minutes and may be compute- or I/O-intensive. - A user must never read another user's file or report. - The same completed file should not be analyzed twice for the same analysis version. ### Clarifying Questions to Ask - What file formats, maximum sizes, and validation rules are supported? - Must analysis begin before upload completes? - What completion latency, concurrency, retention, and availability targets apply? - Can reports be regenerated under newer analysis versions? ### What a Strong Answer Covers - Direct multipart object-storage uploads with authenticated control-plane APIs - Durable metadata and an explicit upload and analysis state machine - Idempotent event publication, queued work, retries, leases, and dead-letter handling - Streaming or chunked processing and independently scalable worker pools - Tenant authorization, malware checks, encryption, lifecycle cleanup, and observability ### Follow-up Questions 1. How would you resume a 200 GB upload after a client loses connectivity? 2. What prevents an object-store event and a client completion call from creating two jobs? 3. How would you support a new analysis version without overwriting an earlier report? ```hint Separate data transfer from orchestration Let object storage carry file bytes while application services manage identity, state transitions, authorization, and asynchronous work. ```