Evaluate Carousel and Billboard Lift
Company: Pinterest
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
You are the product data scientist supporting Pinterest. Two analytics prompts were described:
1. **Carousel feature at the top of the home feed.** Pinterest wants to add a horizontally scrollable carousel at the top of the app, similar to Stories. The module could either resurface Pins the user previously saved or show new recommended Pins. How would you:
- clarify the product goal,
- define primary success metrics and guardrail metrics,
- design an online experiment,
- and decide whether the feature should launch?
Suppose the carousel is launched in an experiment and the **overall home-page CTR decreases**. Explain how you would determine whether this is a true product regression versus a denominator effect, cannibalization of the existing vertical feed, ranking-quality issues, novelty effects, segmentation effects, or logging problems.
2. **Billboard experiment.** Pinterest also runs an offline billboard campaign in a subset of cities or DMAs. How would you estimate the causal lift of the campaign on app activity? Discuss the choice of experimental or quasi-experimental design, unit of randomization, key metrics, power considerations, and how you would handle spillovers, seasonality, and overlapping national marketing campaigns.
Quick Answer: This question evaluates experiment design, causal inference, metric definition, diagnostic analysis, and measurement validity within the Analytics & Experimentation domain for a Data Scientist role.