How would you design a Shop Ads ranking algorithm?
Company: Meta
Role: Data Scientist
Category: Machine Learning
Difficulty: easy
Interview Round: Technical Screen
## Context
An ads ranking system serves ads via an auction. You want to **uprank Shop Ads** relative to Website Ads to improve user conversion and help certain advertisers.
## Task
Design (at a high level) an algorithmic approach for ranking that incorporates the “Shop Ads” preference.
Cover:
1. **Objective function**: What are you optimizing (e.g., expected value, user utility, revenue)? How do you handle multiple stakeholders (users, advertisers, platform)?
2. **Modeling approach**: What predictions do you need (CTR, CVR, purchase value)? Any special considerations for Shop Ads vs Website Ads?
3. **How to incorporate uprank** without breaking the auction (e.g., boost factor, constraint, or multi-objective optimization).
4. **Training data & labels**: delayed conversions, missing data (privacy), selection bias from prior ranking.
5. **Offline + online evaluation**: what offline metrics, counterfactual evaluation methods, and online tests would you run?
6. **Key risks**: feedback loops, calibration issues, fairness across advertiser sizes, cold start.
You may assume the platform currently ranks ads by something like `score = bid * pCTR * pCVR * value` (or a learned equivalent).
Quick Answer: This question evaluates a candidate's understanding of machine learning-driven ad ranking, auction mechanics, multi-stakeholder objective formulation, predictive modeling for CTR/CVR/value, and measurement challenges such as delayed feedback, selection bias, and fairness in online advertising.