Design and analyze an SBA mini case experiment

A small‑business acquisition team plans a 2‑week pilot: new accounts receive a $100 credit after identity verification. Primary KPI: 30‑day activation rate = % of new accounts that achieve ≥$50 net ad spend within 30 days of signup. Guardrails: refund rate, support tickets per 1,000 accounts, average first‑30‑day spend, and CAC. Baseline activation is 12%. Budget allows 10,000 treatment accounts over 14 days; daily traffic and mix vary by region and industry. Design a rigorous experiment: choose randomization unit and stratification variables; handle day‑of‑week and regional seasonality; plan for non‑compliance and fraud. Compute the required sample size for 80% power and α=0.05 to detect a +2.0 percentage‑point lift; state all assumptions and show your calculation method. Specify the exact analysis plan: ITT vs. TOT definitions, variance reduction (e.g., CUPED with pre‑verification activity), guardrail monitoring with kill‑switch thresholds, and a precise decision rule (e.g., one‑sided test with a superiority margin and multiplicity control). Describe how you will check data quality (event lag, duplicate accounts, bots), measure heterogeneity of treatment effects (industry, region, spend propensity), and ensure results generalize beyond the 2‑week window (novelty and seasonality adjustments). Finally, outline how you would recommend rollout if the observed lift is +1.6pp with a 95% CI of [+0.2pp, +3.0pp] while support tickets increase by 12%.