Design an ad recommendation and ranking system
Company: Meta
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
You are building an **ad recommendation/ranking** system for a content feed (e.g., short-form videos). At each feed position, you may show either an organic item or an ad.
Design the end-to-end approach:
1) Define the **objective(s)** (e.g., revenue, user experience) and how you would combine them (constraints vs. weighted objective).
2) Propose modeling choices for predicting key events (CTR, CVR, p(revenue), long-term value), including labels, features, and handling delayed feedback.
3) Explain how you would address common issues:
- selection bias / counterfactual evaluation (logged policy)
- exploration vs. exploitation
- cold start (new ads, new users)
- advertiser budget/pacing constraints
- fairness/diversity constraints (optional but discuss if relevant)
4) Provide an evaluation plan: offline metrics + online A/B metrics and guardrails.
Quick Answer: This question evaluates end-to-end ad recommendation and ranking design skills, covering machine learning modeling, recommender systems, causal and counterfactual evaluation, online learning, and production ML engineering within the Machine Learning domain, at a high-level system-design and modeling abstraction.