Design and evaluate a fraud detection strategy

Context

You are interviewing for a Fraud Data Scientist role at a payments company. The company has a fraud model and some operational constraints.

Part A — Fraud domain knowledge

1. What are common fraud types in payments? Give examples.
2. What is Account Takeover (ATO) and how does it typically happen end-to-end?
3. Explain first‑party vs. third‑party fraud and how they differ in incentives, signals, and labeling.

Part B — Model evaluation

You have a fraud model that outputs a risk score per transaction.

1. Which metrics would you use to measure model quality (e.g., precision/recall/ROC‑AUC/PR‑AUC/cost)?
2. How would you choose a production threshold when false positives create customer friction?
3. What pitfalls exist in evaluating fraud models (delayed labels, selection bias, feedback loops, changing base rates, etc.)?

Part C — Strategy from scratch

Describe how you would design a fraud strategy from 0 given:

• You can use a combination of rules + model scores + manual review .
• Review capacity is limited.
• The business cares about both fraud loss and customer experience .

Part D — Case prompt

A one‑pager summary says:

• Current system intercepts only ~40% of fraud (low fraud capture).
• Fraud loss is high.
• A large share of fraud is coming from emerging regions .
• You have limited resources to ship large engineering changes.
• If the strategy causes very low precision (example: precision drops toward ~2% on blocked/flagged events), complaints will increase .

Task: Propose a practical, staged plan to reduce fraud loss. Include:

• Primary metric(s), diagnostic metric(s), and guardrails.
• How you would segment (regions, customer types, payment methods, etc.).
• What actions you would take (thresholding, rules, step‑up auth, review routing).
• How you would validate impact and monitor after launch.