Design load balancing, caching, and idempotent APIs

You are designing a backend service that will be deployed across multiple instances behind a load balancer. Answer the following:

1) Load balancing

• What is a load balancer and why is it used?
• Compare common routing policies:
  • round-robin / weighted round-robin
  • least connections
  • consistent hashing
  • latency/health-based routing
  • sticky sessions (session affinity)
• Discuss L4 vs L7 load balancing and when you’d choose each.

2) Caching

• Where can caching be applied (client/CDN, edge, reverse proxy, application, database)?
• Explain cache read/write patterns (cache-aside, read-through, write-through, write-back).
• How would you handle invalidation, TTLs, and staleness?
• Describe how to prevent or mitigate cache stampede/thundering herd.

3) Idempotency support

Design an API/system that supports idempotent requests (e.g., payment charge, order creation).

• Define idempotency and why it matters.
• Propose an idempotency-key design (where the key comes from, how long it lives).
• Explain how the service stores and checks idempotency state.
• Address concurrency (same key arrives simultaneously), retries, and partial failures.
• Mention how idempotency interacts with load balancing and distributed deployments.