Design a machine learning system for a fintech data-connectivity platform such as **Plaid** that detects fraudulent activity across the user lifecycle — bank-account linking, identity verification, and suspicious financial behavior. The system must flag high-risk users, connections, and transactions in **near real time** so the platform can take a graded action (allow, step-up verification, manual review, or block) while keeping the **false-positive rate low** enough that legitimate users are rarely interrupted. Produce an end-to-end design that covers: (1) the fraud scenarios you target and how you frame the prediction problem, (2) data sources and label construction, (3) features (including relational/graph signals), (4) the modeling approach, (5) the offline training pipeline and the online serving architecture, (6) offline and online evaluation, and (7) handling of delayed labels, class imbalance, adversarial behavior, privacy, and monitoring. ```hint Frame the target first "Is this user a fraudster" is probably too coarse a label to score directly. Think about *which decision point* the platform actually controls and could act on — and what you'd condition the prediction on at that moment. How does the output need to be shaped so it can drive more than a hard allow/block? ``` ```hint Where does the signal live before you reach for a model? Some of the cheapest, most robust signals in fraud aren't model outputs at all. What can you compute about *rates* of behavior over short and long time windows, and about the *track record* of the entities involved (device, IP, email, phone, bank account)? Consider what has to be true at decision time for those signals to be usable. ``` ```hint Fraud is rarely a lone actor Fraudsters reuse infrastructure — devices, IPs, phone numbers, bank accounts — and often operate in coordinated groups. What representation lets you exploit those shared connections, and what kinds of features fall out of it? Think about how much modeling machinery you actually need to extract that value, and in what order you'd add it. ``` ```hint Labels and leakage are the traps When is an event truly "legitimate," given that fraud is confirmed only after some delay? That question constrains how you define negatives, how you split train/test, and which signals you're allowed to use as features. Separately: if the system blocks the riskiest users, what happens to the labels you'd have learned from? ``` ### Constraints & Assumptions - **Scale (assume for estimation):** millions of end users across thousands of bank/partner integrations; tens of millions of account-link attempts and transaction events per day; thousands of QPS at peak on the scoring path. - **Latency budget:** the link/transaction flow blocks on the risk decision, so online scoring must return in roughly **tens to low-hundreds of milliseconds** (p99). - **Labels are delayed and noisy:** confirmed fraud (chargebacks, partner reports, account closures) arrives days to weeks after the event, is highly imbalanced (positives are a small fraction of a percent), and some fraud is never reported. - **Adversarial setting:** fraudsters actively probe and adapt; they share devices/IPs/accounts and operate in rings. - **Regulated data:** financial PII is sensitive; decisions may need reason codes for compliance and must avoid disparate impact on protected groups. ### Clarifying Questions to Ask - Which decision point are we scoring first — the bank-link attempt, identity verification, or the transaction — and what is the per-decision latency budget? - What graded actions can the system take (allow / step-up / review / block), and who consumes the score (automated policy, risk ops, partners)? - What ground-truth labels are actually available (chargebacks, partner fraud reports, account closures, manual reviews), and what is the typical label-maturation lag? - What are the relative business costs of a missed fraud (loss) versus a blocked legitimate user (friction/churn) versus a manual review? - Are there regulatory requirements for reason codes, fairness, or restrictions on which attributes we may use? - What is the expected fraud base rate, and how adversarial/fast-moving are the patterns we have seen so far? ### What a Strong Answer Covers - **Problem framing:** explicit prediction target(s) tied to a decision point, and the mapping from a calibrated score to graded actions. - **Data & labels:** concrete sources, a principled positive/negative label definition with maturation windows, and honest treatment of delay, sparsity, and selection bias. - **Features:** identity/account, velocity, device/network, behavioral, transaction, and relational/graph groups — with attention to time-consistency. - **Modeling:** a justified modeling approach with a clearly chosen primary scorer, a sensible role for cheaper deterministic logic and for unsupervised/relational signals, and calibrated outputs. - **Systems:** an offline training pipeline (feature store, time-based splits, leakage avoidance) and a low-latency online path with fallbacks. - **Evaluation:** imbalance-aware offline metrics and business/online metrics; a rollout plan (shadow → canary → full). - **Hard realities:** delayed labels, class imbalance, adversarial drift, privacy/compliance, and monitoring/retraining triggers. - **Tradeoffs:** rules vs. ML vs. graph; precision/recall vs. friction; complexity vs. operability. ### Follow-up Questions - A fraud ring shares one device across 50 freshly created accounts that each link a different bank. How would your graph features and velocity counters catch this, and how quickly? - Your offline PR-AUC improved but online fraud loss did not drop. What are the likely causes (label maturation, distribution shift, feedback loop, threshold/calibration), and how would you diagnose it? - How do you evaluate and threshold a model whose own blocks suppress the labels you would have used to measure it? Describe a strategy to keep an unbiased signal. - The model starts flagging a legitimate immigrant population at a higher rate because of IP/geolocation features. How do you detect and remediate this disparate impact without gutting recall?