Design a File Archive System With End-to-End Data Integrity Guarantees
Company: Bloomberg
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
Design a file archive system: clients archive files to keep them for a long time and retrieve them later. The interview focused on data integrity. An archived file must never be silently lost or corrupted, and a retrieval must return exactly the bytes that were archived or fail loudly. The report gives only this focus, so file sizes, volume, retention periods and access patterns are open: agree on them before designing.
```hint Where bytes can go wrong
List every place a file's bytes pass through or rest between the client's upload and a retrieval years later, and ask how corruption at each one would be noticed.
```
```hint Data nobody reads
Think about a file that nobody retrieves for years. How would you know it is still intact before someone needs it?
```
### Clarifying Questions
- How large are files, how much data arrives per day, and how fast does the total grow?
- How long must files be kept, and may they ever be modified or deleted, for example after a retention period or under a legal hold?
- How quickly must a retrieval complete: seconds, or are minutes to hours acceptable?
- What durability is expected, and across which failure domains: disks, racks, zones, regions?
- Must integrity be provable to an outside auditor, or only detectable internally?
### What a Strong Answer Covers
- Requirements, scale and a durability target agreed up front
- End-to-end checksums from the client through storage and back, verified at every hop
- Redundancy across failure domains, through replication or erasure coding, and a commit protocol that acknowledges only durable data
- Background scrubbing that finds silent corruption, and automatic repair
- Consistent metadata: manifests, versioning, immutability and safe deletion
- Protection against software bugs, operator error and tampering, plus integrity monitoring
### Follow-up Questions
- A software bug wrote bad data for a week, and the server computed its checksums on the bad data. How does the design limit the damage and recover?
- How do you migrate the archive to new storage hardware without risking integrity?
- How would you prove to an auditor that a file retrieved today is the file archived years ago?
- Deduplication would save space. What does it do to integrity and to deletion?
Overview: A system design question about a long-term file archive in which data integrity is the main concern: archived files must never be silently lost or corrupted and must read back byte for byte. It tests end-to-end checksums, redundancy, scrubbing and repair, immutability, safe deletion and tamper evidence.