Transactional Banking System Design

What's being tested

The interviewer is probing your ability to design a transactional account-balance service that is correct under concurrency, auditable, and resilient at scale. Expect to show data modeling (ledger vs materialized balance), concurrency control (locking, optimistic CAS, isolation), API-level idempotency, and pragmatic choices for replication/partitioning and failure recovery. Capital One cares because money correctness, latency, and auditability are non-negotiable; interviewers want to see principled tradeoffs and implementation-level details a software engineer would own.

Core knowledge

• Ledger-first model: store an immutable transaction journal where balance = sum(credits) − sum(debits). This supports auditability, reconciliation, and simplifies concurrency compared to storing only a mutable balance.

• Materialized balance cache: keep a denormalized balance for fast reads, updated atomically from ledger writes via transactional update or idempotent upserts; reconcile periodically with ledger to detect drift.

• ACID transactions and isolation: prefer serializable or at least repeatable read for balance-affecting ops; for high throughput, use optimistic concurrency with version/CAS to reduce locking contention.

• Idempotency at API layer: require client idempotency-key persisted with result; ensure server-side de-duplication (unique constraint) and TTL for keys to bound storage.

• Exactly-once vs at-least-once: implement idempotency/de-duplication for effective exactly-once semantics on business operations while accepting lower-level at-least-once delivery on transport.

• Distributed consistency models: single-leader/shard for a given account gives strong local consistency; for multi-node replication, quorum formula is N = 2f + 1 to tolerate f failures (use majority writes/reads).

• Cross-service flows: avoid distributed 2PC where possible; use saga or compensating transactions for long-running multi-system operations, persisting state machine events to resume after failures.

• Partitioning/sharding: shard by account-id (hash or customer-id) so writes for one account are single-shard, reducing cross-shard transactions; monitor hot-shard skew and implement split/migrate tools.

• Latency & batching: batch non-critical writes (e.g., analytics CDC) but keep balance-affecting paths low-latency (target p99 SLA); measure and tune WAL/fsync vs latency tradeoffs.

• Reconciliation & anti-entropy: schedule background jobs that recompute balance from ledger and compare to materialized value; surface reconciliation drift and provide automatic correction or human alerting.

• Failure recovery & WAL: use durable write-ahead log or append-only store (Postgres WAL, Kafka/event log) to rebuild state; ensure checkpoints and snapshots to bound recovery time.

Tip: design the minimal invariant you must enforce (e.g., balance never negative unless overdraft allowed), implement checks at the transaction boundary, and make them testable.

Worked example — Design a highly reliable account balance system

First 30 seconds: ask clarifying questions — required throughput, latency SLAs, allowed consistency (strong vs eventual), support for multi-currency/overdraft, audit/regulatory retention. Declare assumptions: per-account write rate moderate (≤100 ops/sec), strong consistency needed for single-account reads/writes, global cross-account transfers supported.

Skeleton answer pillars: (1) ledger-first data model: append-only transactions table with unique transaction-id and metadata; (2) materialized balance per account updated atomically using a conditional update (version/CAS) in the same DB transaction as the ledger insert; (3) API idempotency using client idempotency-key stored with transaction row to dedupe retries; (4) replication/sharding: shard by account, single-writer per shard for simplicity, replicate via leader-follower with majority quorum; (5) background reconciliation and monitoring with alerts for divergences.

A concrete tradeoff: choosing single-shard single-writer gives strong local consistency and simple serializability, but limits cross-account atomic transfers — you must either perform transfer in the same shard or implement distributed saga/compensating steps. I would explicitly flag avoiding 2PC in favor of a transfer coordinator maintaining idempotent steps and compensations.

Close: state what you'd prototype (schema, example SQL/CAS snippets, idempotency table), add metrics (p99 latency, reconciliation drift counts), and say "if I had more time I'd add automated shard-splitting, chaos-testing of leader failover, and formal proofs/benchmarks of no-double-spend under simulated restarts."

A second angle — Design a scalable banking system

Here the emphasis shifts to global scale, multi-tenant isolation, external integrations (ATMs, payment rails), and operational practices. Reframe pillars: (1) horizontal sharding strategy with account affinity and dynamic rebalancing, (2) service boundaries — separate card/authorization, clearing/settlement, and ledger services with well-defined event contracts, (3) use asynchronous event-driven flows (Kafka CDC) for downstream systems while keeping core balance updates synchronous and idempotent. You'd discuss global replication for disaster recovery (active-passive or multi-region with conflict-free design), routing of external requests to the responsible shard, throttling and circuit-breakers for external payment rails, and capacity planning for peak loads (e.g., payroll day). The same primitives—ledger, idempotency, reconciliation, and sharded single-writer guarantees—apply but at a larger operational and latency/bandwidth envelope.

Common pitfalls

Pitfall: Modeling only a mutable balance without a transaction ledger. This makes auditing, reconciliation, and replay after failures extremely difficult; auditors expect an append-only trail and regulators require transaction-level retention.

Pitfall: Reaching for distributed 2PC as the first solution for cross-account transfers. 2PC adds blocking, complex recovery, and operational fragility; interviewers prefer saga patterns or designing transfers to be single-shard where possible.

Pitfall: Ignoring idempotency and de-duplication. Naively retrying on timeouts without server-side dedupe leads to double-posts. The stronger answer stores idempotency-key in a dedupe table with unique constraint and returns the previous outcome if seen.

Connections

Interviewers may pivot to consensus protocols (e.g., Raft, Paxos) for leader election and replication choices, or to observability (SLOs, p99 latency, audit logs) and chaos-testing (failure injection). They may also ask for a migration plan from monolith to microservices stressing data consistency.

Further reading

• Designing Data-Intensive Applications — Martin Kleppmann — excellent on logs, replication, and consistency tradeoffs.

• Stripe on Idempotency — practical patterns for idempotent APIs and client-server coordination.