Design fraud detection from raw transactions evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
You receive a large multi-table dataset of transactions and customer/merchant metadata with delayed or partial fraud labels. Design an end-to-end system to decide whether each transaction is fraudulent. Cover feature engineering (velocity, graph/link features, device/IP signals), handling class imbalance and label latency, training/validation splits that prevent leakage, thresholding for review capacity, real-time scoring and latency budgets, feedback loops from manual review/chargebacks, monitoring (drift, TPR/FPR, approval rate), and a backtesting plan.
Overview: Design fraud detection from raw transactions evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
System Design: End-to-End Transaction Fraud Detection
Context
You are given a large, multi-table dataset of transactions and customer/merchant metadata. Fraud labels arrive with delays (e.g., chargebacks weeks later) and may be partial (e.g., only for reviewed or disputed transactions). Design an end-to-end system to decide, in real time, whether to approve, decline, or send each transaction to manual review.
Requirements
Cover the following aspects with clear assumptions and rationale:
Data and Feature Engineering
Velocity features (multi-horizon counts/sums/uniques).
Graph/link features across entities (user/card/email/device/IP/merchant).
Device and IP signals (fingerprinting, geolocation, proxy/TOR, ASN risk).
Labels, Class Imbalance, and Latency
Handling severe class imbalance.
Handling delayed/partial labels and selective-label bias.
Training and Validation Splits
Splitting to avoid leakage in time and across entities.
Ensuring offline/online feature parity.
Decision Thresholding and Review Capacity
Approve/Decline/Review policy with cost-sensitive thresholds.
Meeting a fixed manual review capacity.
Real-Time Scoring and Latency Budgets
Online feature retrieval and model serving under strict latency.
Fallbacks and degradation strategies.
Feedback Loops
Incorporating manual review outcomes and chargebacks.