Design and Evaluate a Home Carousel
Quick Overview
This question evaluates a candidate's expertise in product analytics, experimentation design, metric definition, and instrumentation for measuring the impact of a new UI feature.
Pinterest
Jan 14, 2026, 12:00 AM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0
Pinterest is considering adding a horizontally scrollable carousel at the top of the Home feed, similar to Instagram Stories. The carousel may surface recently saved Pins, recommended Pins, or topic collections.
As a Data Scientist, explain how you would evaluate this feature from product, analytics, and experimentation perspectives.
Your answer should address:
- What product goal(s) you would first clarify, and how those goals would change your evaluation framework.
- What primary, secondary, and guardrail metrics you would define. Be explicit about trade-offs among metrics such as module CTR, overall Home CTR, saves, downstream engagement, session depth, retention, and ad or monetization impact.
- What events, dimensions, and logging you would instrument to build the analytical system for this feature.
- How you would design an A/B test or staged rollout, including the randomization unit, exposure definition, novelty effects, power / minimum detectable effect, and possible sources of bias or confounding.
- Suppose the feature launches and overall Home-page CTR decreases. What are the most plausible explanations, and how would you distinguish among a logging bug, a denominator effect, cannibalization of the existing feed, segment-specific effects, or a true product regression?
Solution
Show