How would you evaluate upranking Shop ads?
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
Meta is considering **upranking ads that send users to an in-app Shop experience** (for example, Facebook/Instagram Shops) relative to ads that send users to an advertiser's external website.
You are asked to evaluate whether this is a good product and business idea.
Please discuss:
- What the product hypothesis is and which stakeholders are affected: users, small advertisers, large omnichannel advertisers, and Meta.
- What the main benefits and risks are. For example, small advertisers may benefit because they can sell directly on-platform without building a full website, while large brands may prefer traffic to their own site or even offline store visits.
- How you would define success metrics and guardrail metrics. Include tradeoffs among user experience, advertiser outcomes, and platform revenue.
- How you would design an A/B test to measure the impact of upranking Shop ads.
- What sources of bias or confounding could mislead you if you only looked at observational data.
- How you would decide whether to launch globally, launch only for certain segments, or not launch at all.
Quick Answer: This question evaluates product analytics, experimentation design, causal inference, and multi-stakeholder trade-off analysis involving users, small advertisers, large omnichannel advertisers, and the platform.