Design a cloud file storage and collaboration service similar to Google Drive or Dropbox. At a high level, the service must let users store files in a hierarchical folder structure, share them with fine-grained permissions, and keep multiple clients (web, desktop, mobile) in sync. Your design should cover the following capabilities: - Upload, download, delete, move, and rename files and folders. - A hierarchical folder structure scoped per user or per organization. - Sharing files and folders with users or groups using roles such as **owner**, **editor**, **commenter**, and **viewer**, with inheritance down a folder subtree. - Large-file uploads that are **chunked and resumable** (survive a dropped connection). - File **version history** and the ability to restore a prior version. - **Synchronization** across web, desktop, and mobile clients, including offline edits that reconcile on reconnect. - **Search** by file name and basic metadata (owner, type, modified time). - High availability, durability, and security. Walk through the end-to-end design: high-level architecture, the public API surface, the split between metadata and blob storage, the metadata/data model, the upload and download paths, the synchronization protocol, the consistency model, the scaling approach, failure handling, and the major trade-offs you would call out. ```hint Where to start The single most important decision is to **separate metadata from blob content**. Metadata (the namespace tree, permissions, versions) lives in a transactional store and is small; file bytes live in object storage and are large and immutable. Almost every other decision follows from this split. ``` ```hint Large files For resumable uploads, think about splitting a file into fixed-size **chunks**, uploading each chunk independently (ideally direct-to-object-storage via pre-signed URLs), checksumming per chunk, and committing a **manifest** of chunk keys as an atomic "complete" step. Content-addressing chunks by hash gives you dedup and idempotent retries for free. ``` ```hint Sync Don't have clients diff their whole tree on every poll. Maintain a **monotonic per-drive change log** (sequence numbers / cursor); a client sends its last cursor and receives an ordered delta. This is the backbone of both incremental sync and conflict detection. ``` ```hint Scale & consistency Reason separately about the two stores. Metadata wants **strong consistency** within a drive (rename/move/permission must be transactional) → partition by drive/workspace so each shard is a single-writer transactional unit. Blobs want **durability + cheap reads** → replicated object storage + CDN for immutable versioned blobs. Push everything non-critical (search indexing, thumbnails, virus scan, notifications) onto an async event bus. ``` ### Constraints & Assumptions - **Users:** tens of millions of registered users; assume a few million daily-active clients each maintaining a persistent sync connection. - **Data:** petabytes of stored bytes; individual files from a few KB to several GB. - **Traffic:** read-heavy on download/list/sync (assume reads dominate writes by roughly an order of magnitude), but sustained write traffic from uploads and edits. - **Durability target:** very high for file content (object stores commonly advertise ~11 nines of annual durability via cross-zone replication); plan to match that. - **Availability:** the metadata and download paths must stay available; brief degradation of search or thumbnails is acceptable. - **Security:** TLS in transit, encryption at rest, short-lived signed access to blobs, and an audit trail for share/download events. - Out of scope unless raised: real-time co-editing of a single document (operational transforms / CRDTs) — treat that as a separate collaboration system; you only need last-writer / conflict-copy semantics for whole-file sync. ### Clarifying Questions to Ask - Is the unit of isolation a single user's drive, or are there shared **organization/team** drives with their own permission roots? This changes the sharding key. - What is the expected **file-size distribution** and the largest file we must support? It drives the chunk size and whether we need multipart at all. - Do we need **block-level delta sync** (Dropbox-style: only re-upload changed chunks of an edited file), or is whole-file re-upload acceptable for v1? - How strong are the **sharing/permission** requirements — simple ACLs, or nested groups, public links, expiring links, and link passwords? - Is **server-side full-text search** (indexing file contents, not just names) in scope, or only metadata search? - What are the **retention** rules for deleted files and old versions (trash window, version count cap, legal hold)? - Do we need **multi-region active-active** writes (global low-latency, e.g. for an enterprise customer like a law firm operating across offices), or is single-region-write with regional read replicas sufficient? ### What a Strong Answer Covers A strong answer is judged on the following dimensions (these are the areas to demonstrate, not the answers themselves): - **Metadata/blob separation** — recognizing the two stores have different consistency, durability, cost, and scaling needs, and designing each accordingly. - **Namespace & data model** — a workable representation of the folder tree (parent pointers, stable node IDs, uniqueness within a parent), versions, and permissions; and how move/rename/delete map onto it. - **Resumable upload path** — chunking, direct-to-storage uploads, per-chunk checksums, an atomic commit/manifest step, and idempotency on retry. - **Download path** — authorization, short-lived signed URLs, CDN caching of immutable versions. - **Sync protocol** — the change-log/cursor mechanism, delta delivery, push vs. poll, and offline-edit reconciliation. - **Permission model** — direct vs. inherited ACLs, group/org membership, conflict resolution, and how to evaluate this fast under read-heavy load (caching, precomputation). - **Consistency model** — being explicit about what is strongly consistent (namespace/permission ops within a shard) vs. eventually consistent (search, previews, notifications), and why. - **Scaling** — partitioning metadata by drive/workspace, object-storage + CDN for blobs, async event bus for derived work, and where the hot spots and bottlenecks are. - **Reliability & security** — durability strategy, backups/PITR, idempotency keys, encryption, signed access, audit logging, and observability (which metrics/SLOs). - **Trade-offs** — articulating the real tensions (strong vs. eventual consistency, direct vs. proxied upload, inheritance vs. precomputed ACLs, dedup vs. simplicity/privacy, single- vs. multi-region writes). ### Follow-up Questions - A user **moves a 50,000-file folder** into a new parent that has different sharing. How do you make the move appear atomic, and how do you propagate the permission change without rewriting 50,000 rows synchronously? - Two desktop clients edit the **same file while both are offline**, then both reconnect. Walk through exactly what your sync protocol does — how the conflict is detected and what the user sees. - How would you add **block-level delta sync** so that editing one page of a 2 GB file re-uploads only a few chunks? What changes in the data model and the upload path? - Storage cost is dominated by **duplicate content** across users (e.g. the same PDF shared widely). How do you deduplicate safely, and what are the privacy and security pitfalls of cross-user dedup? - A whole region hosting the **metadata primary** goes down. What is your failover story, what is the RPO/RTO, and what could a client observe as lost or stale during the cutover?