Design a File Archive System With End-to-End Data Integrity Guarantees

Quick Overview

A system design question about a long-term file archive in which data integrity is the main concern: archived files must never be silently lost or corrupted and must read back byte for byte. It tests end-to-end checksums, redundancy, scrubbing and repair, immutability, safe deletion and tamper evidence.

Design a File Archive System With End-to-End Data Integrity Guarantees

Company: Bloomberg

Role: Software Engineer

Category: System Design

Difficulty: medium

Interview Round: Technical Screen

Design a file archive system: clients archive files to keep them for a long time and retrieve them later. The interview focused on data integrity. An archived file must never be silently lost or corrupted, and a retrieval must return exactly the bytes that were archived or fail loudly. The report gives only this focus, so file sizes, volume, retention periods and access patterns are open: agree on them before designing. ```hint Where bytes can go wrong List every place a file's bytes pass through or rest between the client's upload and a retrieval years later, and ask how corruption at each one would be noticed. ``` ```hint Data nobody reads Think about a file that nobody retrieves for years. How would you know it is still intact before someone needs it? ``` ### Clarifying Questions - How large are files, how much data arrives per day, and how fast does the total grow? - How long must files be kept, and may they ever be modified or deleted, for example after a retention period or under a legal hold? - How quickly must a retrieval complete: seconds, or are minutes to hours acceptable? - What durability is expected, and across which failure domains: disks, racks, zones, regions? - Must integrity be provable to an outside auditor, or only detectable internally? ### What a Strong Answer Covers - Requirements, scale and a durability target agreed up front - End-to-end checksums from the client through storage and back, verified at every hop - Redundancy across failure domains, through replication or erasure coding, and a commit protocol that acknowledges only durable data - Background scrubbing that finds silent corruption, and automatic repair - Consistent metadata: manifests, versioning, immutability and safe deletion - Protection against software bugs, operator error and tampering, plus integrity monitoring ### Follow-up Questions - A software bug wrote bad data for a week, and the server computed its checksums on the bad data. How does the design limit the damage and recover? - How do you migrate the archive to new storage hardware without risking integrity? - How would you prove to an auditor that a file retrieved today is the file archived years ago? - Deduplication would save space. What does it do to integrity and to deletion?

Overview: A system design question about a long-term file archive in which data integrity is the main concern: archived files must never be silently lost or corrupted and must read back byte for byte. It tests end-to-end checksums, redundancy, scrubbing and repair, immutability, safe deletion and tamper evidence.

Bloomberg logo
Bloomberg
Nov 12, 2026
mediumSoftware EngineerTechnical ScreenSystem Design
0
0

Design a file archive system: clients archive files to keep them for a long time and retrieve them later. The interview focused on data integrity. An archived file must never be silently lost or corrupted, and a retrieval must return exactly the bytes that were archived or fail loudly. The report gives only this focus, so file sizes, volume, retention periods and access patterns are open: agree on them before designing.

Clarifying Questions Guidance

  • How large are files, how much data arrives per day, and how fast does the total grow?
  • How long must files be kept, and may they ever be modified or deleted, for example after a retention period or under a legal hold?
  • How quickly must a retrieval complete: seconds, or are minutes to hours acceptable?
  • What durability is expected, and across which failure domains: disks, racks, zones, regions?
  • Must integrity be provable to an outside auditor, or only detectable internally?

What a Strong Answer Covers Guidance

  • Requirements, scale and a durability target agreed up front
  • End-to-end checksums from the client through storage and back, verified at every hop
  • Redundancy across failure domains, through replication or erasure coding, and a commit protocol that acknowledges only durable data
  • Background scrubbing that finds silent corruption, and automatic repair
  • Consistent metadata: manifests, versioning, immutability and safe deletion
  • Protection against software bugs, operator error and tampering, plus integrity monitoring

Follow-up Questions Guidance

  • A software bug wrote bad data for a week, and the server computed its checksums on the bad data. How does the design limit the damage and recover?
  • How do you migrate the archive to new storage hardware without risking integrity?
  • How would you prove to an auditor that a file retrieved today is the file archived years ago?
  • Deduplication would save space. What does it do to integrity and to deletion?