Evaluate Product-Ranking Algorithm with Precision and Recall Metrics

Quick Overview

This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Evaluate Product-Ranking Algorithm with Precision and Recall Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Evaluate Product-Ranking Algorithm with Precision and Recall Metrics

Company: Meta

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Onsite

##### Scenario Instagram Shopping wants to improve its product-ranking algorithm. ##### Question Which offline and online metrics would you use to evaluate a ranking model for a shopping feed and why? Explain how precision and recall translate to this setting and how you would reconcile offline metrics with on-site business KPIs. ##### Hints Discuss NDCG, MAP, click-through, conversion lift and user-level funnels.

Quick Answer: This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Evaluate Product-Ranking Algorithm with Precision and Recall Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Machine Learning/Meta
Meta logo
Meta
Aug 4, 2025, 10:55 AM
mediumData ScientistOnsiteMachine Learning
5
0

Evaluate Product-Ranking Algorithm with Precision and Recall Metrics

Scenario

Instagram Shopping wants to improve its product‑ranking algorithm for the shopping feed. The goal is to select and order products for each user to maximize value (e.g., conversions/revenue) while maintaining a good user experience.

Task

  1. Propose offline metrics to evaluate a ranking model for a shopping feed and explain why they fit this use case.
  2. Propose online (experiment) metrics, including how you would structure user‑level funnels.
  3. Explain how precision and recall translate to ranking in this setting (including @K variants).
  4. Explain how you would reconcile offline metrics with on‑site business KPIs.

Hints (for scope)

  • Consider NDCG and MAP for offline ranking evaluation.
  • Consider click‑through, conversion lift, and funnels for online evaluation.

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the task, data shape, labels, constraints, and evaluation metric.
  • State assumptions behind the math or modeling technique you choose.
  • Connect theory to practical training, debugging, and deployment implications.

What a Strong Answer Covers Guidance

  • Correct definitions and formulas where the prompt requires them.
  • A practical explanation of how the method behaves on real data.
  • Trade-offs, failure modes, diagnostics, and mitigation strategies.
  • Evaluation choices that match the product or modeling objective.

Follow-up Questions Guidance

  • How would noisy labels, class imbalance, or distribution shift affect the answer?
  • What would you monitor after deployment?
  • Which baseline would you compare against first?
Loading comments...

Browse More Questions

More Machine Learning•More Meta•More Data Scientist•Meta Data Scientist•Meta Machine Learning•Data Scientist Machine Learning