Evaluate New Feed-Ranking Algorithm with A/B Testing
Company: Pinterest
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
##### Scenario
A social-media company wants to evaluate a new feed-ranking algorithm intended to increase daily active minutes.
##### Question
a) Formulate the A/B test hypothesis (null and alternative) and select primary and guardrail metrics.
b) Determine the minimal detectable effect and required sample size for 95% power, two-tailed α = 0.05.
c) After launch, the product dashboard shows a time-series chart of average active minutes by group. Describe what you look for to confirm experiment health (e.g., parallel pre-period, no data loss) and how you would interpret a sudden mid-test dip.
d) Explain how you would establish causal inference if rollout is geography-based rather than randomized.
##### Hints
Cover randomization, practical significance, and difference-in-differences when random assignment is impossible.
Quick Answer: Pinterest experimentation prompt on evaluating a new feed-ranking algorithm, covering A/B hypotheses, daily active minutes, guardrails, sample size, MDE, experiment health, mid-test dips, and geo-based causal inference.