Evaluate Facebook's Restaurant Recommendations Feature Effectiveness
Overview
This question evaluates a candidate's competency in online experimental design and product analytics, including defining treatment and control groups, selecting primary and guardrail metrics, estimating sample size and duration, and planning rollout and decision criteria.
Meta
Jul 12, 2025, 6:59 PM
Data Scientist
Onsite
Analytics & Experimentation
7
0
Experiment Design: Restaurant Recommendations in Facebook News Feed
Context
Facebook is considering launching restaurant recommendation units inside the News Feed (e.g., a card showing nearby restaurants with images, ratings, and a "Learn more/Order" CTA). The goal is to drive meaningful user engagement and downstream restaurant orders without harming overall feed health.
Task
Design an experiment to evaluate this feature. Specify:
- Treatment vs. control definition and unit of randomization
- Success metrics (primary and secondary) with clear definitions
- Guardrail metrics to protect feed health and revenue
- Sample size and duration approach (with assumptions and example calculations)
- Rollout and decision criteria, including ramp strategy and validation
Solution
Login required
Show