PracHub
QuestionsPremiumLearningGuidesInterview PrepNEWCoaches
|Home/Analytics & Experimentation/Other

Design an A/B test for a Celebrate reaction

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's competency in experimental design for friend-based social feeds, including defining business-aligned success metrics, handling network interference, powering tests, planning heterogeneity analyses, and specifying diagnostics and follow-up experiments.

  • hard
  • Other
  • Analytics & Experimentation
  • Data Scientist

Design an A/B test for a Celebrate reaction

Company: Other

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

WeChat Moments considers adding a 'Celebrate' reaction. 1) Define success metrics that credibly tie to business outcomes (e.g., 7/30/90‑day retention, posting rate) and guardrails (e.g., session length, hide/mute/report rates, feed quality). 2) Choose the randomization unit (user, post, user→post edge) and defend it under network interference; propose a cluster or switchback design to mitigate spillovers. 3) Power the test (state effect size assumptions) and specify a staged rollout with sequential monitoring that controls Type I error. 4) Plan heterogeneity analysis (e.g., by country, age, new vs. existing users). 5) Detect novelty and saturation effects; propose ramp, cooldown, and difference‑in‑differences diagnostics. 6) Define a follow‑up experiment to ensure the feature doesn’t incentivize low‑quality posting; include pre‑registered decision rules for ship/iterate/rollback.

Quick Answer: This question evaluates a data scientist's competency in experimental design for friend-based social feeds, including defining business-aligned success metrics, handling network interference, powering tests, planning heterogeneity analyses, and specifying diagnostics and follow-up experiments.

Related Interview Questions

  • Design metrics resilient to data quality - Other (Medium)
  • Design and power an A/B on question mix - Other (medium)
  • Design an A/B test with guardrails - Other (hard)
  • Find and fix metric drops systematically - Other (medium)
  • Separate demand from supply for jeans - Other (medium)
Other logo
Other
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Analytics & Experimentation
1
0

Experiment Design: Adding a "Celebrate" Reaction to WeChat Moments

Context

WeChat Moments is a friend-based social feed with posting, viewing, and reactions. The product team is considering adding a new "Celebrate" reaction alongside existing reactions. You are asked to design an experiment that credibly measures business impact while handling network interference.

Assume you can log impressions, reactions, hides/mutes/reports, friend-graph edges, notifications, and sessions. You can randomize at user, post, or user→post edge level and can also randomize by clusters (e.g., geos or graph partitions) or by time (switchback).

Tasks

  1. Define success metrics that credibly tie to business outcomes (e.g., 7/30/90-day retention, posting rate) and guardrails (e.g., session length, hide/mute/report rates, feed quality).
  2. Choose the randomization unit (user, post, or user→post edge) and defend it under network interference; propose a cluster or switchback design to mitigate spillovers.
  3. Power the test (state effect size assumptions) and specify a staged rollout with sequential monitoring that controls Type I error.
  4. Plan heterogeneity analysis (e.g., by country, age, new vs. existing users).
  5. Detect novelty and saturation effects; propose ramp, cooldown, and difference-in-differences diagnostics.
  6. Define a follow-up experiment to ensure the feature doesn’t incentivize low-quality posting; include pre-registered decision rules for ship/iterate/rollback.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Other•More Data Scientist•Other Data Scientist•Other Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.