Decide whether to launch Group Story
Overview
This question evaluates a data scientist's competence in experimental design and causal inference, covering metric definition and guardrails, network interference and randomization choices, statistical treatment of heavy-tailed engagement metrics, power and duration planning, incremental lift decomposition for cannibalization, and cohort heterogeneity analysis. It is commonly asked in Analytics & Experimentation interviews because it probes both conceptual understanding of causal and interference issues and practical application of statistical experiment methodology at product-launch scale.
A new Group Story feature may cannibalize regular stories but increase overall engagement. Propose the experiment and decision framework: 1) Identify success metrics and guardrails: feature adoption, overall session time, regular-story posts, creator activity, and crash rate. 2) Address network effects: choose between user-cluster randomization (e.g., communities) versus switchback at the feed level; justify to minimize interference. 3) Powering heavy-tailed session lengths: choose variance-stabilizing transforms, winsorization, or quantile treatment effects; justify. 4) Measure cannibalization vs net new creation; define incremental lift decomposition. 5) Duration and ramp strategy with pre-period CUPED. 6) Heterogeneity analysis by cohort (new vs existing creators) and by groups size; specify how results would change your launch/no-launch decision and what holdout you would keep for long-term effects.