Diagnose Discrepancy in A/B Test Conversion Rate Results

##### Scenario An e-commerce firm plans to send personalized marketing emails to increase purchase conversions and wants to rigorously evaluate the impact. ##### Question Design an A/B test to measure whether personalized emails lift conversion rate. Specify the primary and guardrail metrics, statistical test, minimum detectable effect, required sample size and duration. After launching on the full user base, a new director reruns the test and sees only a 2 % lift versus the original 20 %. List possible causes and the analyses you would run to diagnose the discrepancy. ##### Hints Think through experiment design, power analysis, instrumentation issues, seasonality, user overlap, novelty effects and segmentation cuts.