PracHub
QuestionsCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Pinterest

Evaluate New Feed-Ranking Algorithm with A/B Testing

Last updated: Mar 29, 2026

Quick Overview

Pinterest experimentation prompt on evaluating a new feed-ranking algorithm, covering A/B hypotheses, daily active minutes, guardrails, sample size, MDE, experiment health, mid-test dips, and geo-based causal inference.

  • medium
  • Pinterest
  • Analytics & Experimentation
  • Data Scientist

Evaluate New Feed-Ranking Algorithm with A/B Testing

Company: Pinterest

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Onsite

##### Scenario A social-media company wants to evaluate a new feed-ranking algorithm intended to increase daily active minutes. ##### Question a) Formulate the A/B test hypothesis (null and alternative) and select primary and guardrail metrics. b) Determine the minimal detectable effect and required sample size for 95% power, two-tailed α = 0.05. c) After launch, the product dashboard shows a time-series chart of average active minutes by group. Describe what you look for to confirm experiment health (e.g., parallel pre-period, no data loss) and how you would interpret a sudden mid-test dip. d) Explain how you would establish causal inference if rollout is geography-based rather than randomized. ##### Hints Cover randomization, practical significance, and difference-in-differences when random assignment is impossible.

Quick Answer: Pinterest experimentation prompt on evaluating a new feed-ranking algorithm, covering A/B hypotheses, daily active minutes, guardrails, sample size, MDE, experiment health, mid-test dips, and geo-based causal inference.

Related Interview Questions

  • How would you evaluate a carousel launch? - Pinterest (medium)
  • How to evaluate a new Carousel feature - Pinterest (easy)
  • Evaluate Fresh Content and Video Experiments - Pinterest (medium)
  • Design and Evaluate a Home Carousel - Pinterest (medium)
  • Evaluate Carousel and Billboard Lift - Pinterest (medium)
|Home/Analytics & Experimentation/Pinterest

Evaluate New Feed-Ranking Algorithm with A/B Testing

Pinterest logo
Pinterest
Jul 12, 2025, 6:59 PM
mediumData ScientistOnsiteAnalytics & Experimentation
81
0

A/B Test a New Feed-Ranking Algorithm

A social-media company wants to evaluate a new feed-ranking algorithm intended to increase daily active minutes per user.

Constraints & Assumptions

  • Use a randomized A/B test when user-level randomization is available.
  • If rollout is geography-based, discuss causal inference alternatives.
  • Daily active minutes are likely skewed and may require variance reduction or robust checks.
  • Cover hypothesis, metrics, sample size, experiment health, and interpretation of time-series behavior.

Clarifying Questions to Ask

  • Is the ranking change static during the experiment, or does it learn online?
  • Is the primary metric minutes per active user-day, per assigned user, or per session?
  • What minimum lift is worth shipping?
  • Are there network effects or creator-side spillovers?

What a Strong Answer Covers

  • Hypotheses for the A/B test and a primary metric such as daily active minutes per user.
  • Guardrails: retention, DAU/MAU, crash rate, latency, negative feedback, content diversity, ads revenue, and integrity reports.
  • Sample-size formula for two-sample mean comparison, using alpha, power, standard deviation, and minimal detectable effect.
  • Discussion of practical significance, two-tailed alpha, and 95% power.
  • Experiment health checks: SRM, covariate balance, pre-period parallel trends, logging completeness, data loss, treatment exposure, and overlapping experiments.
  • Diagnosis of a mid-test dip: outage, logging issue, traffic mix shift, release, novelty/fatigue, external event, or ramp change.
  • Geo-based rollout analysis using difference-in-differences, event study, synthetic control, pre-trend checks, and cluster-robust uncertainty.

Follow-up Questions

  • How would you choose the MDE?
  • What if average minutes rise but negative feedback also rises?
  • Why might geography rollout violate parallel trends?
  • How would CUPED reduce sample size?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Pinterest•More Data Scientist•Pinterest Data Scientist•Pinterest Analytics & Experimentation•Data Scientist Analytics & Experimentation

Write your answer

Your first approved answer each day earns 20 XP.

Sign in to write your answer.
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • AI Coding Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.