Trust, Safety, Fake Accounts, And Fraud

What's being tested

Interviewers probe a candidate's ability to design, evaluate, and interpret detection systems for fraud / fake accounts and abuse, balancing statistical rigor with product constraints. Expect questions on metric selection under extreme class imbalance, experiment design when actions cause network interference, and choosing operating points given asymmetric costs and limited human-review capacity. Meta cares because small mis-calibrations cascade into user experience harm, enforcement cost overruns, or blind spots exploited by adversaries.

Core knowledge

• precision and recall definitions and tradeoff: precision = TP/(TP+FP), recall = TP/(TP+FN); always report both for asymmetric-cost problems.

• PR-AUC vs ROC-AUC: PR-AUC is preferred under heavy class imbalance because it focuses on positive-class performance; ROC-AUC can be misleading when negatives dominate.

• precision@k and review-budget metrics: use precision@k or precision at fixed review capacity when human reviews or enforcement budget constrain actioning.

• F_β and cost-weighted metrics: choose β to reflect relative cost (β>1 emphasizes recall). Alternatively optimize explicit expected cost: E[Cost] = FP·c_FP + FN·c_FN.

• Calibration and reliability: use reliability diagrams and Brier score to check predicted probabilities; good calibration enables risk-based thresholding and downstream decisioning.

• Labeling bias & sampling: enforcement-derived labels are biased (policy, visibility). Use stratified sampling, importance weighting, or adjudicated audits to estimate true rates and correct evaluation.

• Threshold selection: tune on time-split validation that mirrors production distribution; use cost function or precision@k for operating point; validate via offline replay and small holdout rollouts.

• Concept drift & adversarial drift: continuously monitor population metrics, feature distributions, and model performance; maintain a plan for periodic retraining and adversarial-feature hygiene.

• Experimentation under interference: for metrics affected by network effects, use cluster randomization, partial-interference assumptions, or graph-based assignment; account for intraclass correlation (ICC) when computing sample sizes.

• Power and minimum detectable effect: compute sample size with inflated variance for clustering: $n_{cluster} \approx n_{ind} \cdot (1 + (m-1)\rho)$ where (m) cluster size and (\rho) is ICC.

• Evaluation under skewed prevalence: when positives are rare, use targeted labeling (oversample suspected positives) and reweight evaluation to produce unbiased performance estimates.

• Downstream impact metrics: measure not only classification metrics but also user-level outcomes (engagement, retention, complaint rate) to detect unintended harms from false positives.

Worked example — Choose Metrics for Evaluating Fake-User Classifier

Frame: first confirm the business objective (reduce fake-user prevalence? reduce scams? prevent policy evasion?), available signals (device, IP, behavioral), labeling process (human takedowns vs synthetic), and operational constraints (daily reviewer capacity, acceptable false-positive rate). Skeleton answer pillars: 1) Offline evaluation: report PR-AUC, precision@k, recall, and F_β tuned to business cost; 2) Thresholding & deployment: pick operating point by optimizing expected cost or review-capacity-constrained precision; 3) Label & sampling strategy: correct for label bias via adjudicated audits and importance weighting; 4) Monitoring: calibration checks, drift detection, and a human-audit loop. Tradeoff flagged: optimizing for high precision reduces customer friction but leaves many fakes undetected; explicitly quantify cost of missed fakes vs cost of wrongful removals and select β or cost weights accordingly. Close: if more time, propose an offline replay experiment and a small randomized holdout rollout to measure real downstream effects (complaints avoided, DAU impact), plus periodic adversarial red-teaming.

A second angle — Design Messenger spam experiment with clustering

Here the constraint is network interference: messages propagate across connections so user-level randomization causes spillovers. Start by clarifying unit (conversation, sender, or recipient), the policy intervention (e.g., aggressive spam filter), and measurement window. Use cluster randomization at the conversation-thread or social-graph-community level to limit interference, compute sample sizes inflated by ICC, and pre-specify network-aware metrics (e.g., conversation-level spam rate, cross-cluster leakage). Analytical tools change: estimate direct vs spillover effects (e.g., two-stage randomization or SUTVA-violation-aware estimators). The same evaluation principles apply — choose metrics aligned to cost (false blocking of legitimate messages vs missed spam) and monitor for behavior changes and adversarial adaptation.

Common pitfalls

Pitfall: Using ROC-AUC as the primary metric under extreme class imbalance. This often hides poor positive-class performance; PR-AUC and precision@k reveal practical effectiveness.

Pitfall: Omitting production constraints when choosing thresholds. Saying "maximize F1" without stating reviewer capacity or cost per FP/FN leads to inactionable recommendations.

Pitfall: Treating enforcement labels as ground truth. Ignoring label bias (suspensions reflect detection, not prevalence) will overestimate recall and give misleading signals about model improvements.

Connections

Interviewers may pivot to feature engineering for fraud (behavioral time-series features, sessionization), causal impact of enforcement (does account removal change network behavior?), or MLE/engineering concerns like online calibration and safety guardrails, so be prepared to connect evaluation choices to downstream deployment and monitoring.

Further reading

• Saito & Rehmsmeier, “The precision-recall plot is more informative than the ROC plot” (2015) — empirical and theoretical reasons to prefer PR under class imbalance.

• Kohavi et al., “Practical Guide to Controlled Experiments on the Web” (MSR) — experiment design fundamentals and pitfalls, including interference and variance issues.