Experiment Design: Adding a "Celebrate" Reaction to WeChat Moments

Context

WeChat Moments is a friend-based social feed with posting, viewing, and reactions. The product team is considering adding a new "Celebrate" reaction alongside existing reactions. You are asked to design an experiment that credibly measures business impact while handling network interference.

Assume you can log impressions, reactions, hides/mutes/reports, friend-graph edges, notifications, and sessions. You can randomize at user, post, or user→post edge level and can also randomize by clusters (e.g., geos or graph partitions) or by time (switchback).

Tasks

1. Define success metrics that credibly tie to business outcomes (e.g., 7/30/90-day retention, posting rate) and guardrails (e.g., session length, hide/mute/report rates, feed quality).
2. Choose the randomization unit (user, post, or user→post edge) and defend it under network interference; propose a cluster or switchback design to mitigate spillovers.
3. Power the test (state effect size assumptions) and specify a staged rollout with sequential monitoring that controls Type I error.
4. Plan heterogeneity analysis (e.g., by country, age, new vs. existing users).
5. Detect novelty and saturation effects; propose ramp, cooldown, and difference-in-differences diagnostics.
6. Define a follow-up experiment to ensure the feature doesn’t incentivize low-quality posting; include pre-registered decision rules for ship/iterate/rollback.