Design an Image Upload Service That Stores Identical Images Only Once

Design an image upload service. Users upload images, list and view their own images, and delete them. When different users upload the same image, the service must store it only once. After the high-level design, the interviewer goes deep on two areas: exactly how an upload reaches storage, and the edge cases of deleting images, in particular how to make sure an image is removed promptly once the last user who has it deletes it. ### Clarifying Questions - What counts as "the same image": byte-identical files only, or also resized and recompressed copies? - How many uploads arrive per day, how large are images on average and at most, and how often are they read? - When the last user who has an image deletes it, how quickly must the bytes disappear, and is there a legal or privacy deadline? - Can one user hold the same image twice, for example in two albums, and does each copy count separately? - Must it be impossible for a user to learn whether someone else has already uploaded a given image? ### Part 1 — Core design and data model Propose the components and the data model. Explain how the service recognizes that two uploads are the same image, and what each user's upload record points to. ```hint Two identities Separate what identifies the bytes from what identifies one user's copy, and ask which of the two two independent uploads would share. ``` #### What This Part Should Cover - Where image bytes and metadata live, and why they are kept apart - How identical content is detected, including the hash choice and its collision risk - Tables for shared content and for per-user records, with their keys and indexes - Uniqueness when two users upload the same image at the same moment ### Part 2 — The upload path Walk through an upload in detail, from the client to durable storage, including large files and failures along the way. ```hint Follow the bytes Decide whether the bytes pass through your servers, who computes the hash and when it can be trusted, and what a half-finished upload leaves behind. ``` #### What This Part Should Cover - Direct uploads to the object store, and resumable uploads for large files - Whether a hash supplied by the client can be trusted, and what skipping the upload of a known hash would expose - The commit step that makes an upload visible, and the cleanup of abandoned uploads - What a duplicate upload costs in bandwidth and in storage ### Part 3 — Deletion edge cases Users delete their images. Cover the edge cases, and explain in particular how you make sure an image is deleted promptly once the last user who has it deletes it. ```hint Race the last delete Picture the last owner deleting the image at the same moment another user uploads identical bytes, and decide what each request must see. ``` #### What This Part Should Cover - Tracking how many users still reference an image, and keeping that count correct - Prompt deletion on the main path versus a periodic cleanup job as a safety net - The race between a last delete and a new identical upload - Every other copy of the bytes: thumbnails, caches, object versions, replicas and backups ### What a Strong Answer Covers - Requirements and rough scale agreed before the design - A content-addressed design whose invariants hold under concurrency, retries and partial failures - Security: knowing an image's hash never grants access to it, and no request reveals whether an image already exists - A cleanup strategy that keeps references, metadata and the object store consistent over time - Monitoring of the deduplication ratio, upload failures and the delay between the last delete and actual removal ### Follow-up Questions - Two users upload the same large image at the same moment. Walk through what each request sees and what ends up stored. - A client offers only the hash of an image and asks to skip the upload because the service already has it. Why is that risky, and what would you allow instead? - How would you detect near-duplicates such as resized or recompressed copies, and would you still store them only once? - After a bug, the stored reference counts no longer match the actual references. How do you detect and repair the drift?