##### Question PayPal is planning to launch a new **homepage feature** (for example, a new CTA module, personalized content, or a redesigned layout) and wants to decide whether to ship it. Design an experiment to evaluate the feature. Assume this is a consumer-facing homepage experience for logged-in users (justify explicitly if you change scope), and that you can instrument homepage events and downstream funnel events (login, add payment method, send money, checkout, etc.). Your answer should cover: 1. **Objective and hypothesis** — State the experiment objective and the null vs alternative hypothesis. 2. **Experiment design** — Propose an A/B test design: unit of randomization (and why it is appropriate), eligibility, assignment consistency, duration considerations, and key validity checks. 3. **Metrics** — Propose and clearly define (numerator/denominator, event definition, time window): - one **primary (success) metric**, - **secondary / diagnostic funnel metrics**, - **guardrail metrics**. 4. **Metric tradeoffs** — Discuss the tradeoffs among candidate metrics. For example, the feature might increase homepage engagement but also slow the page, distract users from core payment flows, or change fraud risk. 5. **Sample size** — Describe how you would determine the required sample size, including the inputs needed (baseline rate/variance, minimum detectable effect, significance level, power, traffic allocation, clustering, etc.). 6. **Increase power** — List practical ways to increase statistical power if traffic is limited, without compromising validity, and explain the tradeoffs. 7. **Pitfalls** — Call out common pitfalls such as novelty effects, seasonality, user heterogeneity (Simpson's paradox), logging errors, and interference across users or devices. 8. **Ship decision** — Define a decision rule for whether to ship.