Design a Globally Distributed Key–Value Configuration Service

Background

You are asked to design a globally distributed key–value configuration service powering thousands of microservices across multiple regions. The service should provide low-latency reads, strong safety for updates, and global availability.

Requirements

• Define the key schema (namespacing, composability, versioning).
• Define the value schema (serialization format, validation, schema evolution).
• Explain partitioning/sharding, indexing, and storage engine choices (e.g., LSM-tree vs B-tree).
• Describe write/read paths and hot-key mitigation.
• Specify the consistency model (linearizable vs eventual), leader election, replication/quorums, and handling of network partitions and failover.
• Describe support for transactions, watches/subscribe for updates, TTLs, and backup/restore.
• Provide capacity planning, SLAs, multi-region replication strategy, and testing approaches.

Assumptions

• Read-heavy (≫95% reads), small values (0.5–8 KB), small key space (≤ few million keys), but very high fanout of clients.
• Writes are rare but correctness-critical; clients expect fast propagation of changes.
• Multi-tenant namespaces, multi-region footprint, thousands of services, tens of thousands of instances.