A marketplace plans to change its search ranking to favor nearby merchants. You will run a 14-day 50/50 user-level randomized A/B test with daily DAU ≈ 200,000 and baseline conversion p0 = 0.12. Expected relative lift = +5% on conversion; guardrails: cancellation rate must not increase by >0.2 percentage points and average delivery time must not worsen by >3 minutes. (a) Compute the minimum per-arm sample size for conversion using a two-sided z-test, α = 0.05, power = 0.80. Show formulas and numeric steps assuming independent Bernoulli trials, then discuss how user clustering and repeat sessions/orders would inflate variance and how you’d correct (e.g., variance inflation factor via empirical design effect or cluster-robust SEs). (b) Specify primary, secondary, and guardrail metrics; pre-register hypotheses; define the decision rule combining effect size and statistical significance (include MDE and non-inferiority thresholds for guardrails). (c) Describe a CUPED or pre-period covariate adjustment using user-level 28-day pretest conversion propensity; provide the adjusted estimator and how you would validate variance reduction (A/A test, placebo checks). (d) Outline a ramp plan (1%→10%→50%→100%), novelty and learning effects monitoring, weekday/seasonality controls, and how to handle attribution and tracking delays (48h late events). (e) If the test is geo-split (city-level) instead of user-split, propose a difference-in-differences setup with city fixed effects and calendar effects; list assumptions and how you’d test for pre-trend balance. (f) Explain how you will monitor and correct for peeking and multiple metrics (alpha-spending, O’Brien–Fleming; FDR for many guardrails) and define a rollback plan.