System Design: Personal Finance Aggregation Platform (Mint-like)
Context
Design a consumer-facing platform that aggregates a user's financial data across banks, credit cards, loans, and other institutions. The system ingests balances and transactions, normalizes data, categorizes spending, supports budgeting, and provides alerts. Assume a modern technical screen context and aim for a pragmatic, production-ready design.
Assumptions (minimal):
-
Scale: 1–5M users, each with 2–6 linked accounts; daily ingestion volume in the tens of millions of transactions.
-
Data freshness: near-real-time via webhooks where available; otherwise periodic polling (e.g., hourly to daily).
-
Regions: initially single region with future multi-region readiness.
Requirements
-
API/ingestion: choose aggregator vs. direct connections; authenticate via OAuth/Open Banking where possible; legacy OFX if needed.
-
Data processing: normalization and deduplication across sources; consistent signs and timestamps.
-
Categorization and budgeting: rules/ML categorization; budgets by category with rollovers and sharing.
-
Alerts/notifications: thresholds, anomalies, upcoming bills; delivery via push/email/in-app.
-
Reconciliation and idempotency: pending-to-posted transitions, retries, and exactly-once semantics.
-
Privacy/security: encryption, tokenization, consent, PCI scope minimization.
-
Scalability/storage: service boundaries, data stores, partitioning, and caching.
-
Failure/latency handling: third-party API slowness/outages, rate limits, backoff, and graceful degradation.
-
Data model: consistent schema supporting multi-currency and recurring bills/subscriptions.