Design an end-to-end fraud detection system. Specify positive/negative labeling strategy given delayed and scarce fraud confirmations, sampling to address extreme class imbalance, feature sets (behavioral, graph, device, merchant), model choices and justification, real-time scoring architecture and latency constraints, thresholding and precision/recall trade-offs, evaluation metrics (PR-AUC, precision@k, cost-sensitive metrics), and monitoring for drift and adversarial adaptation.

This question evaluates a Machine Learning Engineer's competency in end-to-end ML system design for real-time payments fraud detection, including labeling under delayed confirmations, handling extreme class imbalance and sampling, feature engineering across behavioral, graph, device and merchant signals, model selection for latency and scale, and production scoring and monitoring architecture. It is commonly asked in the ML System Design category to assess how an engineer balances low-latency decision-making with delayed sparse labels, calibration and threshold trade-offs, operational scalability and resiliency, and drift/adversarial detection, testing both conceptual understanding and practical application.

How do I approach ML System Design interview questions?

ML System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master ml system design interviews.

What difficulty level is this interview question?

This is a hard difficulty ML System Design question, commonly asked during Onsite rounds at PayPal.

What role is this question designed for?

This question is commonly asked for Machine Learning Engineer candidates at PayPal during technical interviews.

Design a traditional fraud detection system

Design an End-to-End Real-Time Payments Fraud Detection System

Context: You are designing a fraud detection system for a large-scale online payments platform. Decisions must be made synchronously at authorization time with tight latency budgets, while confirmed fraud labels (e.g., chargebacks) arrive late and are scarce.

Specify and justify the following:

Labeling strategy under delayed, scarce confirmations

How to define positive/negative labels when chargebacks arrive weeks later.
Aging/observation windows, handling disputed outcomes, and avoiding target leakage.

Sampling to handle extreme class imbalance

Offline training strategies (downsampling, weighting) and how to keep calibration.
Online serving considerations.

Feature sets

Behavioral/velocity features.
Graph/link features across users, devices, payment instruments.
Device/network features.
Merchant/context features.

Model choices and justification

Baseline and advanced models suitable for latency and scale.
Handling graphs, sequences, and semi-/weak supervision.

Real-time scoring architecture and latency constraints

Event ingestion, online/offline feature store, streaming aggregations, model serving, and fallbacks.
Expected P99 latency budget and resiliency.

Thresholding and precision/recall trade-offs

Decision policies (approve/review/decline) using cost-aware thresholds and calibration.

Evaluation metrics

PR-AUC, precision@k, expected-cost/profit metrics, and how to evaluate with delayed labels and policy bias.

Monitoring for drift and adversarial adaptation

Detecting data/model drift, label delay proxies, and adversarial pattern monitoring.
Retraining cadence, rollout, and guardrails.

Design an End-to-End Real-Time Payments Fraud Detection System

Specify and justify the following:

Labeling strategy under delayed, scarce confirmations

How to define positive/negative labels when chargebacks arrive weeks later.
Aging/observation windows, handling disputed outcomes, and avoiding target leakage.

Sampling to handle extreme class imbalance

Offline training strategies (downsampling, weighting) and how to keep calibration.
Online serving considerations.

Feature sets

Behavioral/velocity features.
Graph/link features across users, devices, payment instruments.
Device/network features.
Merchant/context features.

Model choices and justification

Baseline and advanced models suitable for latency and scale.
Handling graphs, sequences, and semi-/weak supervision.

Real-time scoring architecture and latency constraints

Event ingestion, online/offline feature store, streaming aggregations, model serving, and fallbacks.
Expected P99 latency budget and resiliency.

Thresholding and precision/recall trade-offs

Decision policies (approve/review/decline) using cost-aware thresholds and calibration.

Evaluation metrics

PR-AUC, precision@k, expected-cost/profit metrics, and how to evaluate with delayed labels and policy bias.

Monitoring for drift and adversarial adaptation

Detecting data/model drift, label delay proxies, and adversarial pattern monitoring.
Retraining cadence, rollout, and guardrails.

Design a traditional fraud detection system

Quick Overview

Design an End-to-End Real-Time Payments Fraud Detection System

Solution

Comments (0)

Design a traditional fraud detection system

Quick Overview

Design an End-to-End Real-Time Payments Fraud Detection System

Solution

Comments (0)