How would you A/B test first trade rate?

Q: How would you A/B test first trade rate?

This question evaluates A/B testing and experimentation design skills—metric definition, unit and randomization choice, threat-to-validity identification, and power/MDE calculation—for a Data Scientist in the Analytics & Experimentation domain.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Coinbase wants to increase new user first trade rate.

Design an experiment (A/B test) to evaluate a product change intended to increase the probability that a newly signed-up user places their first trade.

In your answer, cover:

Hypothesis and what user behavior you are trying to change.
Primary metric (pick a clear definition, e.g., “first trade within 7 days of signup”) and at least 2 secondary/guardrail metrics (risk, compliance, user harm, revenue quality).
Experiment unit & randomization (user/account/device), eligibility rules, and when a user is considered “exposed”.
Key threats to validity (selection bias, learning/novelty effects, interference, instrumentation issues, sample ratio mismatch).
Power/MDE approach (how you would size the test, what baseline inputs you need).
What you would do if the result is not statistically significant but directional, or if key guardrails regress.

How would you A/B test first trade rate?

Quick Overview

Solution

Comments (0)