Context
You work on a brokerage/investing app. The team is considering launching fractional share trading (users can buy/sell fractions of a stock/ETF instead of whole shares).
Task
-
Explain the expected benefits
of fractional shares for the business and for users.
-
Propose an
experimentation plan
to evaluate the launch:
-
What is the
primary success metric
?
-
What
diagnostic metrics
help explain movement?
-
What
guardrail metrics
ensure you don’t harm users or the business?
-
Define the
unit of randomization
(user/account/household/etc.) and the treatment/control experience.
-
Call out key risks: interference/network effects, novelty effects, seasonality, and any selection bias.
-
Sample size / power
-
Show how you would estimate the
required sample size
for the primary metric (state assumptions such as baseline rate/variance, MDE,
α
, and power).
-
If you need to
adjust the sample size mid-experiment
, explain a statistically valid approach.
-
Constraint scenario
-
If the required sample size is
too large
(you don’t have enough traffic or time), what do you do? Provide at least 3 practical options (design, metrics, variance reduction, ramp strategy, or alternative inference methods) and discuss tradeoffs.
Output
Provide a structured written plan, including metric definitions, assumptions used in power/MDE calculations, and a final recommendation on whether/how to launch.