Idempotent API Design

What's being tested

Interviewers are probing whether you can design financial APIs that remain correct when clients retry, networks time out, workers crash, or requests race concurrently. At Coinbase, a Software Engineer must assume that duplicate requests will happen and that “charge twice” or “credit twice” is an unacceptable failure mode. Strong answers show how idempotency, transactional integrity, concurrency control, and auditability work together, especially around ledgers, account balances, transfers, account opening, and delayed rewards like cashback. The interviewer is not looking for a slogan like “use an idempotency key”; they want the storage model, request lifecycle, failure behavior, and tradeoffs.

Core knowledge

• Idempotency means applying the same logical operation multiple times has the same durable effect as applying it once. For write APIs, this usually requires a client-supplied Idempotency-Key, because two identical-looking requests may be different business intents.

• Idempotency keys should be scoped by caller and endpoint, such as (user_id, endpoint, idempotency_key), not globally. A practical table uses key, request_hash, status, response_body, resource_id, created_at, and expires_at, with a unique constraint on the scoped key.

• Request fingerprinting prevents accidental key reuse. Store a normalized request_hash = SHA256(method + path + canonical_json_body); if the same key arrives with a different hash, return 409 Conflict or 422 Unprocessable Entity instead of silently replaying a stale response.

• Atomic write boundaries matter more than the key itself. For a money movement API, insert the idempotency record, ledger entries, and resulting transfer state in one Postgres transaction. If the key is committed but the ledger write is not, retries can become stuck or incorrectly suppressed.

• Double-entry ledgering is the usual Coinbase-grade model for balances. Every transfer creates balanced debits and credits where $\sum entries.amount = 0$ within a transaction. Available balance should be derived from immutable ledger entries or maintained as a transactional projection, not updated as an isolated mutable number.

• Uniqueness constraints are your first line of defense. Use UNIQUE(account_id, idempotency_key) for API dedupe and UNIQUE(transfer_id, account_id, entry_type) for ledger entries. Application checks alone are unsafe under concurrent retries.

• Concurrency control must be explicit. Use row-level locks like SELECT ... FOR UPDATE, optimistic version checks like WHERE version = ?, or serializable transactions. For balance checks, the invariant is usually available_balance >= debit_amount, enforced inside the same transaction as the ledger write.

• Replay semantics should be deterministic. If the original request succeeded, return the same 201 or 200 response body. If it failed validation before side effects, it can safely fail again. If it is still processing, return 202 Accepted, 409 Conflict, or a documented “in progress” response.

• TTL policies are a product and correctness tradeoff, but the engineering default for financial writes is longer retention. Many APIs keep idempotency records for 24 hours to 7 days; ledger systems may retain operation IDs indefinitely because storage is cheap relative to financial risk.

• Asynchronous workflows need idempotent consumers too. If an API creates a transfer_id and publishes to Kafka or a worker queue, every downstream side effect should dedupe on a stable business key like transfer_id, not on delivery attempt. Assume at-least-once delivery.

• External payment rails complicate idempotency. Your internal API may be idempotent, but ACH, card, or bank partners may use their own reference IDs. Persist the external rail_reference_id and reconcile by status transitions such as PENDING, SETTLED, FAILED, and REVERSED.

• Observability should distinguish safe retries from bugs. Track metrics like idempotency hit rate, key conflict rate, duplicate ledger-entry rejection count, transaction retry count, and write-path p99 latency. A spike in conflicts may indicate a client incorrectly reusing keys.

Worked example

For Design a bank account ledger, a strong candidate should start by clarifying the operations: deposits, withdrawals, internal transfers, reversals, and balance reads; whether balances must be strongly consistent; and whether external rails are involved. Then state a core assumption: all money movement must be represented as immutable double-entry ledger entries, and every write API must be idempotent because clients and workers will retry.

The answer can be organized around four pillars. First, define APIs such as POST /transfers requiring Idempotency-Key, with request hashing and replay behavior. Second, define storage: accounts, transfers, ledger_entries, and idempotency_keys in Postgres, with unique constraints around operation identity. Third, describe the transaction flow: validate request, reserve or lock the account row, insert or read the idempotency record, create balanced ledger entries, update transfer state, and commit atomically. Fourth, cover failure handling: if the client times out after commit, the retry returns the stored response; if the transaction aborts, no ledger entries exist and the retry can execute normally.

One tradeoff to flag explicitly is whether to compute balances by summing ledger entries or maintain a cached balance projection. Summing is maximally auditable but too slow for high-volume accounts; a projection is fast but must be updated in the same transaction and periodically reconciled against the immutable ledger. A good close would be: “If I had more time, I’d discuss reconciliation jobs, reversal entries instead of deletes, and how to test crash points between idempotency write, ledger write, and response persistence.”

A second angle

For Design account opening workflow, the same concept applies, but the side effects are not just money movement. The workflow may create a user profile, store PII, call KYC/AML vendors, open an internal account, and send notifications. Here, idempotency should be modeled around a stable application_id or client-supplied key so retries do not create duplicate accounts or duplicate vendor checks.

Unlike a ledger transfer, the workflow may be long-running and partially complete, so a state machine matters: STARTED, PII_COLLECTED, KYC_PENDING, APPROVED, ACCOUNT_OPENED, REJECTED. Each transition should be idempotent and guarded by valid previous states. The candidate should emphasize that a retry should resume or return the current application state, not restart the workflow from scratch.

Common pitfalls

Pitfall: Saying “make POST idempotent by using PUT” without explaining operation identity.

HTTP method semantics are not enough for financial correctness. PUT /accounts/{id} can still race, and POST /transfers can be perfectly safe if it requires an idempotency key, stores the response, and enforces unique ledger operation IDs.

Pitfall: Treating Redis as the only idempotency store.

A cache-only design can lose keys during eviction or failover, allowing duplicate money movement. Redis can help with short-lived locks or fast-path checks, but the durable source of truth for financial idempotency should usually live in the same transactional database as the side effect, such as Postgres.

Pitfall: Ignoring “same key, different payload.”

This is a common interviewer follow-up. The wrong answer is to return the original response no matter what; the better answer is to store a canonical request hash and reject mismatches, because key reuse may represent a client bug or a malicious attempt to mask a different operation.

Connections

Interviewers often pivot from idempotent API design into transaction isolation, distributed locks, outbox pattern, saga orchestration, or ledger reconciliation. They may also ask how you would test the design with concurrent requests, injected timeouts, process crashes, and duplicate message delivery.

Further reading

• Stripe API Idempotent Requests — canonical practical pattern for idempotency keys, replayed responses, and parameter mismatch handling.

• Designing Data-Intensive Applications — Martin Kleppmann — strong background on transactions, consistency, fault tolerance, and distributed system failure modes.

• Outbox Pattern — Chris Richardson — useful for connecting database commits to reliable asynchronous processing without pretending distributed writes are atomic.