Explain fraud types and evaluate a fraud model
Quick Overview
This question evaluates knowledge of payment fraud types, end-to-end account takeover mechanics, labeling and ground-truth issues between first‑party and third‑party fraud, selection of detection signals and evaluation metrics, and the ability to design a rapid production strategy; category/domain: Machine Learning, position type: Data Scientist, abstraction level: applied systems and feature-level modeling with operational deployment considerations. It is commonly asked because payment platforms need to balance fraud loss versus customer friction and operational cost, so interviews probe understanding of attack flows, label bias, ML and business metrics (precision/recall, fraud dollars prevented, false‑positive cost), threshold tradeoffs, and practical plans for data, modeling, decisioning, monitoring, and iteration under tight timelines and label/chargeback delays.
You are interviewing for a Fraud Data Scientist role at PayPal.
Answer the following:
- List common fraud types relevant to payments (e.g., account takeover, first‑party fraud, third‑party fraud, merchant fraud). For each, give a short definition and an example.
- Account Takeover (ATO): Explain how ATO typically happens end-to-end (attacker acquisition → credential takeover → monetization), and what signals/features you would expect to be useful for detection.
-
First‑party vs. third‑party fraud:
- Define each clearly.
- Explain why the label quality/ground truth can differ between them.
- Describe at least one way mislabeling can bias model training/evaluation.
-
How would you measure the effectiveness of a fraud model in production?
- Provide at least 5 metrics, including both ML metrics (e.g., precision/recall) and business/risk metrics (e.g., fraud dollars prevented, false-positive cost).
- Describe the key tradeoffs (e.g., precision vs recall) and how decision thresholds should be chosen.
- Clarify what the “positive class” is and how class imbalance affects evaluation.
- You are asked to design a fraud strategy from scratch (“from 0”). Outline a practical plan across data, modeling, decisioning, monitoring, and iteration. Assume you must deploy something useful within 6–8 weeks.
State any assumptions you need (e.g., time zone, label delay, chargeback window).