Evaluate Fake-Account Classifier with Precision and Recall Metrics

Quick Overview

This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Evaluate Fake-Account Classifier with Precision and Recall Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Evaluate Fake-Account Classifier with Precision and Recall Metrics

Company: Meta

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Onsite

##### Scenario You have built a model that flags fake accounts; leadership wants evidence it works well in production. ##### Question Which evaluation metrics would you choose to judge the fake-account classifier and why? Explain the trade-offs among precision, recall, F1, ROC-AUC, and business costs of false positives versus false negatives. ##### Hints Discuss class imbalance, threshold tuning, and cost-based metric selection.

Quick Answer: This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Evaluate Fake-Account Classifier with Precision and Recall Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Machine Learning/Meta
Meta logo
Meta
Aug 4, 2025, 10:55 AM
mediumData ScientistOnsiteMachine Learning
6
0

Evaluate Fake-Account Classifier with Precision and Recall Metrics

Evaluating a Fake-Account Classifier in Production

Scenario

You have trained a model that flags fake accounts. Leadership wants clear, defensible evidence that it works well in production and understands the trade-offs of using it to take actions (e.g., auto-ban vs. human review).

Task

Recommend the evaluation metrics you would use to judge the fake-account classifier and explain why. Discuss the trade-offs among:

  • Precision, recall, F1
  • ROC-AUC
  • Business costs of false positives (blocking a real user) vs. false negatives (missing a fake)

Include in your answer:

  • How class imbalance affects metric choice
  • How you would pick and tune thresholds
  • How you would incorporate cost-based metric selection

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the task, data shape, labels, constraints, and evaluation metric.
  • State assumptions behind the math or modeling technique you choose.
  • Connect theory to practical training, debugging, and deployment implications.

What a Strong Answer Covers Guidance

  • Correct definitions and formulas where the prompt requires them.
  • A practical explanation of how the method behaves on real data.
  • Trade-offs, failure modes, diagnostics, and mitigation strategies.
  • Evaluation choices that match the product or modeling objective.

Follow-up Questions Guidance

  • How would noisy labels, class imbalance, or distribution shift affect the answer?
  • What would you monitor after deployment?
  • Which baseline would you compare against first?
Loading comments...

Browse More Questions

More Machine Learning•More Meta•More Data Scientist•Meta Data Scientist•Meta Machine Learning•Data Scientist Machine Learning