System Design: End-to-End Transaction Fraud Detection
Context
You are given a large, multi-table dataset of transactions and customer/merchant metadata. Fraud labels arrive with delays (e.g., chargebacks weeks later) and may be partial (e.g., only for reviewed or disputed transactions). Design an end-to-end system to decide, in real time, whether to approve, decline, or send each transaction to manual review.
Requirements
Cover the following aspects with clear assumptions and rationale:
-
Data and Feature Engineering
-
Velocity features (multi-horizon counts/sums/uniques).
-
Graph/link features across entities (user/card/email/device/IP/merchant).
-
Device and IP signals (fingerprinting, geolocation, proxy/TOR, ASN risk).
-
Labels, Class Imbalance, and Latency
-
Handling severe class imbalance.
-
Handling delayed/partial labels and selective-label bias.
-
Training and Validation Splits
-
Splitting to avoid leakage in time and across entities.
-
Ensuring offline/online feature parity.
-
Decision Thresholding and Review Capacity
-
Approve/Decline/Review policy with cost-sensitive thresholds.
-
Meeting a fixed manual review capacity.
-
Real-Time Scoring and Latency Budgets
-
Online feature retrieval and model serving under strict latency.
-
Fallbacks and degradation strategies.
-
Feedback Loops
-
Incorporating manual review outcomes and chargebacks.
-
Exploration/holdout strategies to mitigate bias.
-
Monitoring and Alerting
-
Drift (input/output), TPR/FPR with delayed labels, calibration, approval/decline rates, review queue health.
-
Backtesting Plan
-
Time-ordered replay, off-policy evaluation, simulation of review capacity, metrics and confidence intervals.
State reasonable assumptions if needed and justify key design choices.