API Idempotency And Concurrency Control

What's being tested

Candidates must show practical mastery of idempotency, concurrency control, and state management for HTTP APIs and caches under real-world failure modes. Interviewers look for the ability to specify a clear API contract, pick a consistency model, design deduplication and conflict-resolution mechanisms that scale, and reason about operational concerns (latency, storage, GC, observability). The fitness-for-purpose tradeoffs (optimistic vs pessimistic locking, synchronous vs async dedupe, cache invalidation patterns) are central.

Core knowledge

• Idempotency definition and contract: an operation is idempotent if applying it multiple times has the same effect as once: $f(f(x)) = f(x)$. Use an idempotency-key header and reason about an idempotency window (e.g., 24–72 hours) rather than forever.

• Idempotency-store pattern: persist (idempotency-key → request-fingerprint, response, timestamp). On duplicate key return stored response (HTTP 200/409 depending contract). Tradeoff: storage grows with unique keys; enforce TTL/GC.

• Deduplication strategies: strong (store whole response and check synchronously) vs best-effort (de-duplicate downstream via eventual reconciliation). Strong dedupe gives correctness but adds latency and DB writes per request.

• Optimistic concurrency: use a version/CAS field or WHERE version = v update/upsert pattern to detect and abort concurrent writes. Works well at high throughput and avoids long locks. Example: UPDATE ... SET state=?, version=version+1 WHERE id=? AND version=?.

• Pessimistic locking: use distributed locks (Zookeeper, etcd, Redis RedLock, Postgres advisory locks) for critical sections; higher latency and risk of deadlocks but simpler correctness for complex multi-row updates.

• Database isolation: know serializable, repeatable read, read committed semantics in Postgres and when to rely on DB transactions vs application-level CAS. serializable prevents anomalies but can increase retries.

• Cache consistency patterns: cache-aside with invalidation on write, write-through, and write-behind. For concurrent writers, prefer invalidation plus a short TTL or version-based checks to avoid stale reads.

• Conflict resolution choices: last-writer-wins (by timestamp), merge logic, or application-level arbitration. Always design with monotonic version stamps or vector clocks if multi-master is involved.

• Storage/scale tradeoffs: a simple idempotency table works up to millions of keys per region; beyond that shard by key, compress payloads, or store only hashes and response refs. Enforce TTL: typical window 24–72 hours to bound storage.

• Latency vs consistency: synchronous dedupe and strong locking increase p99 latency; if sub-100ms p99 is required, favor optimistic checks and async recovery pipelines for rare conflicts.

• Observability & SLOs: instrument idempotency-hit-rate, conflict-rate (CAS failures), retry counts, mean/95/99 latencies, and GC lag for idempotency-store. Design alarms for rising conflict or dedupe misses.

• Failure and retry semantics: define API behavior for network failures (client retrying when server returned a 5xx vs timed out). Document whether retrying with same key is expected to be safe; prefer idempotency-key requirement for non-idempotent operations.

Tip: choose a default idempotency window and document it in the API; include key TTL and GC in SLAs so clients know when to regenerate keys.

Worked example — Design an Ad Assignment API

First 30s framing: ask whether assignments are per-user or global, expected QPS and p99 latency targets, acceptable staleness, what constitutes a duplicate (exact same request body or semantic idempotency?), and whether clients can supply an Idempotency-Key. Skeleton answer pillars: (1) API contract — require Idempotency-Key header, return 200 with existing assignment if duplicate; (2) Store design — Postgres assignments table with id, user_id, ad_id, state, version, and an idempotency_keys table mapping key→assignment_id,response,timestamp; (3) Concurrency — use optimistic version updates + DB unique constraints to detect races; (4) Cache & performance — Redis cache-aside for reads, invalidate on write with version stamps; (5) Observability/GC — metrics on dedupe-hit, CAS-failures, and background job to GC idempotency entries after TTL. One tradeoff to flag: synchronous dedupe (check idempotency table before processing) guarantees correctness but adds a DB hit and may increase p99; an alternative is accept occasional duplicate processing and reconcile asynchronously if low-impact. Close by noting next steps: shard idempotency store by client or user, add benchmarks for QPS, and implement chaos tests simulating client retries and partial failures.

A second angle — Handle cache-update conflicts in distributed services

The focus shifts to cache–database consistency: same tools apply (version stamps, CAS, invalidation) but the main constraint is stale reads and high read QPS. Prefer versioned writes: increment a monotonic version in Postgres and propagate (version, payload) to Redis. On cache miss read DB; on write, update DB within a transaction then publish an invalidation message (or write-through to cache). For heavy contention, Redis's WATCH/MULTI provide optimistic atomic updates; for multi-row invariants, use DB transactions and then asynchronous cache invalidation. Another option is write-through with synchronous cache update to avoid invalidation races at the cost of write latency. Instrument stale-hit ratio and tail-latency impact of invalidations.

Common pitfalls

Pitfall: Treating idempotency as a client-only contract. Relying on clients to pick unique keys without server-side storage or checks leads to undetectable duplicates; always persist keys and responses (or hashes) server-side with TTL.

Pitfall: Picking locks by default. Suggesting distributed locks as the first solution ignores latency and availability costs; interviewers prefer optimistic CAS unless multi-row transactional invariants demand pessimistic locking.

Pitfall: Forgetting GC and storage growth. Designing an idempotency store without TTL/compaction leads to unbounded growth; state this up front and propose shard/TTL/compact-hashing or archival strategies.

Connections

Candidates should be ready to pivot to related topics: distributed transactions / sagas for multi-service updates, exactly-once delivery semantics in message systems (Kafka transactions), and observability/chaos testing to validate retry and conflict behavior.

Further reading

• [Designing Data-Intensive Applications — Martin Kleppmann] — chapters on replication, consistency, and distributed transactions are directly relevant.

• Stripe Idempotency Best Practices (blog/docs) — a concrete, industry-standard idempotency-key contract and tradeoffs.