How would you A/B test first trade rate?

Coinbase wants to increase the rate at which newly signed-up retail users place their first trade. Design an A/B test for a product change intended to improve this first-trade rate. Your answer should define the hypothesis, experiment design, metrics, validity checks, power approach, and decision rules. ### Constraints & Assumptions - Treat this as a product analytics or risk analytics interview question. - The product change should improve activation without encouraging fraud, poor user outcomes, or compliance problems. - Use a clear first-trade definition, such as an executed trade within a fixed window after signup. - Assume user-level randomization unless you justify another unit. - You do not need to calculate an exact sample size, but you should explain the inputs required. ### Clarifying Questions to Ask - What product change are we testing: onboarding, education, funding, order entry, or incentives? - Which users are eligible: all new signups or only users who pass identity checks? - What first-trade window matters for the business: 24 hours, 7 days, or 30 days? - Are there risk, fraud, or compliance guardrails that would block launch even if activation improves? ### What a Strong Answer Covers - A concrete hypothesis and activation mechanism. - Primary metric, secondary funnel metrics, and guardrails. - Randomization unit, eligibility, exposure, and analysis population. - Threats to validity such as sample ratio mismatch, selection bias, novelty effects, instrumentation errors, and interference. - Power and MDE reasoning. - A launch, iterate, or rollback decision framework. ### Follow-up Questions - What would you do if the result is positive overall but negative in one high-risk segment? - How would you analyze a directional but not statistically significant result? - How would you distinguish a true activation lift from one-time low-quality trades? - What dashboard would you monitor during ramp-up?