System Design: End-to-End Transaction Fraud Detection

Context

You are given a large, multi-table dataset of transactions and customer/merchant metadata. Fraud labels arrive with delays (e.g., chargebacks weeks later) and may be partial (e.g., only for reviewed or disputed transactions). Design an end-to-end system to decide, in real time, whether to approve, decline, or send each transaction to manual review.

Requirements

Cover the following aspects with clear assumptions and rationale:

1. Data and Feature Engineering
   • Velocity features (multi-horizon counts/sums/uniques).
   • Graph/link features across entities (user/card/email/device/IP/merchant).
   • Device and IP signals (fingerprinting, geolocation, proxy/TOR, ASN risk).
2. Labels, Class Imbalance, and Latency
   • Handling severe class imbalance.
   • Handling delayed/partial labels and selective-label bias.
3. Training and Validation Splits
   • Splitting to avoid leakage in time and across entities.
   • Ensuring offline/online feature parity.
4. Decision Thresholding and Review Capacity
   • Approve/Decline/Review policy with cost-sensitive thresholds.
   • Meeting a fixed manual review capacity.
5. Real-Time Scoring and Latency Budgets
   • Online feature retrieval and model serving under strict latency.
   • Fallbacks and degradation strategies.
6. Feedback Loops
   • Incorporating manual review outcomes and chargebacks.
   • Exploration/holdout strategies to mitigate bias.
7. Monitoring and Alerting
   • Drift (input/output), TPR/FPR with delayed labels, calibration, approval/decline rates, review queue health.
8. Backtesting Plan
   • Time-ordered replay, off-policy evaluation, simulation of review capacity, metrics and confidence intervals.

State reasonable assumptions if needed and justify key design choices.