Design a Read-Only Backend for a Billion Identifiers
Quick Overview
Design a read-only backend for a billion IDs using grounded capacity estimates, suitable partitioning and indexes, caching, replica routing, and versioned data loads.
Design a Read-Only Backend for a Billion Identifiers
Company: Google
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
Design a backend serving read-only data associated with roughly one billion identifiers. Explain data organization, request routing, caching, replication, and the implementation principles behind any storage or cache technology you propose.
### Constraints & Assumptions
The source supplies the identifier scale but not record size, lookup semantics, request rate, or data-update process. Do not equate one billion IDs with a specific petabyte total. Clarify whether requests test membership or retrieve records.
### Clarifying Questions to Ask
- Is each request a point lookup by ID, a range query, or an analytical scan?
- What is the record size and expected hot-key distribution?
- How is the read-only dataset initially loaded or replaced, and how fresh must reads be?
```hint Estimate bytes before choosing a store
The number of identifiers alone does not determine whether the dataset fits in memory or needs many storage shards.
```
### What a Strong Answer Covers
- Capacity estimates based on record/index size and replication, with stated assumptions.
- Partitioning and replica routing matched to access patterns.
- Cache behavior, hot keys, and health-aware load balancing without unnecessary complexity.
- Internal data structures and failure behavior of chosen components.
### Follow-up Questions
- Under uniformly distributed requests, what benefit would a complex adaptive load balancer provide?
- How would you publish a new immutable dataset version without mixing old and new records?
Overview: Design a read-only backend for a billion IDs using grounded capacity estimates, suitable partitioning and indexes, caching, replica routing, and versioned data loads.
Design a backend serving read-only data associated with roughly one billion identifiers. Explain data organization, request routing, caching, replication, and the implementation principles behind any storage or cache technology you propose.
Constraints & Assumptions
The source supplies the identifier scale but not record size, lookup semantics, request rate, or data-update process. Do not equate one billion IDs with a specific petabyte total. Clarify whether requests test membership or retrieve records.
Clarifying Questions to Ask Guidance
Is each request a point lookup by ID, a range query, or an analytical scan?
What is the record size and expected hot-key distribution?
How is the read-only dataset initially loaded or replaced, and how fresh must reads be?
What a Strong Answer Covers Guidance
Capacity estimates based on record/index size and replication, with stated assumptions.
Partitioning and replica routing matched to access patterns.
Cache behavior, hot keys, and health-aware load balancing without unnecessary complexity.
Internal data structures and failure behavior of chosen components.
Follow-up Questions Guidance
Under uniformly distributed requests, what benefit would a complex adaptive load balancer provide?
How would you publish a new immutable dataset version without mixing old and new records?