Choose KPIs for short-video recommendations

Instagram launches a new short‑video recommender. Choose one primary success metric and define it precisely (formula, numerator/denominator, unit of analysis, time window, outlier handling). List at least three guardrail metrics and why. You observe: mean watch time per session +3%, session starts/user −2%. Decide if this ships, and justify with a weighted decision framework. Now design the A/B test: assignment (user- or session-level), novelty warmup, SRM checks, bucketing, holdout length, and seasonality controls. Target MDE is a +2% relative lift on mean watch time per session; baseline mean = 120s, SD = 180s, alpha = 0.05, power = 0.8, 50/50 split, 10M DAU. Estimate required sample size per arm, expected runtime, and key failure modes (e.g., creator-side regressions, regressions in content diversity). Finally, resolve the conflict when 'metric A up, metric B down' by proposing tie-breaker rules and escalation criteria.