Design and evaluate an ads ranking algorithm
Quick Overview
This question evaluates a candidate's proficiency in designing and evaluating production-scale ads ranking systems within Machine Learning, covering ranking architecture, predictive modeling, offline and online evaluation, and operational monitoring.
Meta
Feb 15, 2026, 9:40 PM
Analytics Engineer
Onsite
Machine Learning
1
0
Loading...
Ads ranking algorithm (sponsored content)
You are designing an algorithm to rank ads in a feed/search results page.
Requirements
- Objective: maximize long-term platform value (e.g., revenue) while maintaining good user experience.
- Constraints: low latency, advertisers have budgets/bids, user experience guardrails, and potential policy/fairness constraints.
Questions
- Describe a ranking architecture (candidate generation → scoring → final ranking) and what models you would use.
- What would you predict (e.g., pCTR, pCVR, expected revenue), and how would you combine predictions with bids/budgets?
- How would you handle common issues: position bias, calibration, cold start for new ads, and feedback loops?
- Propose an offline evaluation plan (metrics + validation strategy) and an online testing plan.
- List key monitoring metrics after launch and how you’d detect regressions or fraud/gaming.
Solution
Show