Design a Payment System: Holds and Captures, Batch Charges, Reconciliation, Idempotency

Design a payment system for a company that charges its customers for usage, for example a platform that bills for API or subscription usage through external card processors. The system must support: - **Holds and charges**: placing a hold (an authorization) on a customer's payment method for an estimated amount, then capturing a final amount that may be less than the hold, or releasing the hold. - **Batch processing**: many charges are created in bulk, for example at the end of a billing period, and must be submitted to processors in batches. - **Reconciliation**: the company's records must be matched against the processors' settlement reports, and every mismatch found and resolved. - **Idempotency**: retries anywhere in the system (clients, internal services, processor calls) must never charge a customer twice. ```hint Follow one dollar Trace a single payment from hold to capture to settlement, and ask at each step what is written where, what happens if the process crashes right after that step, and how a retry knows what already happened. ``` ```hint Your records versus theirs Assume the processor's view and yours will disagree sometimes; decide which record is authoritative for what, and how disagreements surface. ``` ### Constraints and Clarifications - Money must never be created or lost: every movement is recorded and auditable. - External processors can time out, return ambiguous errors, or report results late. ### Clarifying Questions - What volumes are expected: payments per day, peak per second, and the size of billing-cycle batches? - Which processors and payment methods (cards, bank transfers), and do holds expire after a processor-defined period? - Are there multiple currencies? - What latency does a user-facing hold need, and what is acceptable for batch captures? - What are the regulatory or compliance requirements for storing card data? ### What a Strong Answer Covers - Explicit payment states and the transitions allowed between them, enforced under concurrency - An auditable source of truth for money movements, and how balances are derived from it - End-to-end protection against duplicate charges, including processor calls whose outcome is unknown - Batch submission that tolerates partial failure and can resume safely - A reconciliation process that detects, classifies and resolves disagreements with processors - Consistency between stored state and emitted events, plus security and monitoring ### Follow-up Questions - A processor call timed out, and you do not know whether the capture happened. What exactly do you do? - How do you handle a hold that expires before the final amount is known? - How would you add a second processor and route payments between them? - How do refunds and chargebacks flow through the ledger and the reconciliation?