Email Targeting Model Experiment Design

You plan to launch a targeting model via email where:

• Treatment: users above a score threshold receive an email
• Control: eligible users are withheld

Answer the following:

1. Metrics
   • Choose one primary success metric and two guardrails (e.g., unsubscribe rate, complaint rate). Justify each.
2. Sample size
   • Baseline 7-day purchase rate: 5%
   • Expected relative lift: 8%
   • Compute the minimum per-arm sample size for a two-sided test with α = 0.05 and 80% power. Show formulas and assumptions. A continuity-corrected normal approximation is acceptable.
3. Ramp and sequential monitoring
   • Propose a traffic ramp plan with sequential monitoring that controls type-I error (e.g., group-sequential or alpha-spending). Specify interim looks and stopping rules.
4. Pre-experiment checks and operational risks
   • Describe checks for randomization, covariate balance, and holdout contamination. Explain how you would handle interference and seasonality around weekends.
5. If pure A/B is infeasible
   • Propose a credible quasi-experimental design (e.g., regression discontinuity on the score threshold or staggered difference-in-differences). List key assumptions you would test and the plots you would include in slides.