Diagnose post-release conversion regression rigorously

On July 15, 2025, version 7.3 was fully rolled out. Daily signup conversion fell from 5.4% (July 1–14 baseline) to 4.9% (July 15–21), while spend and traffic mix appear stable. Design a rigorous analysis to determine whether v7.3 caused the −0.5pp drop and what to do next. Specify:

• Identification strategy: e.g., geo‑based holdout (10% traffic on v7.2) with Difference‑in‑Differences, or synthetic control using pre‑period predictors; include formulas and assumptions (parallel trends, SUTVA).
• Guardrail metrics (latency, crash rate, page load) and falsification checks (invariant metrics like bot share).
• CUPED or pre‑period covariate adjustment to reduce variance; compute MDE given N=3,000,000 sessions/day, α=0.05, power=0.8.
• Segmentation plan (device, country, channel cohorts) with multiple‑testing control (e.g., BH‑FDR) and a pre‑registered decision rule for rollback.
• Sensitivity analyses: day‑of‑week seasonality, exposure dosage, novelty effects, and a regression discontinuity at rollout time.
• Concrete outputs: SQL to build daily cohorts, an experiment notebook outline, and the exact criteria you would use to recommend rollback vs. mitigation.