Design and evaluate an ads ranking algorithm
Company: Meta
Role: Analytics Engineer
Category: Machine Learning
Difficulty: easy
Interview Round: Onsite
## Ads ranking algorithm (sponsored content)
You are designing an algorithm to rank ads in a feed/search results page.
### Requirements
- Objective: maximize long-term platform value (e.g., revenue) while maintaining good user experience.
- Constraints: low latency, advertisers have budgets/bids, user experience guardrails, and potential policy/fairness constraints.
### Questions
1. Describe a **ranking architecture** (candidate generation → scoring → final ranking) and what models you would use.
2. What would you predict (e.g., pCTR, pCVR, expected revenue), and how would you combine predictions with bids/budgets?
3. How would you handle common issues: position bias, calibration, cold start for new ads, and feedback loops?
4. Propose an **offline evaluation** plan (metrics + validation strategy) and an **online testing** plan.
5. List key monitoring metrics after launch and how you’d detect regressions or fraud/gaming.
Quick Answer: This question evaluates a candidate's proficiency in designing and evaluating production-scale ads ranking systems within Machine Learning, covering ranking architecture, predictive modeling, offline and online evaluation, and operational monitoring.