Design experiment and analyze volume drop scenario
Company: Citi
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Technical Screen
Answer the following two prompts as a Data Scientist working on Risk / Product Analytics.
---
## Prompt 1 — Experiment design: increase new-user first trade rate
Coinbase wants to **increase the new-user first trade rate**.
Design an experiment (A/B test) to evaluate a product change intended to increase activation.
Your answer should cover:
1. **Hypothesis** and example treatment(s).
2. **Metric design**
- Primary metric (e.g., “first trade within 7 days of signup”).
- Diagnostic metrics (funnel steps, time-to-first-trade, etc.).
- Guardrails (risk/fraud, customer experience, revenue, platform reliability).
3. **Experiment setup**
- Eligibility criteria (which users are included/excluded).
- Randomization unit (user/device/account) and why.
- Runtime considerations (novelty/learning effects, ramping, SRM checks).
4. **Threats to validity**
- Selection bias, interference/spillovers, delayed conversion, missing data.
5. **If results are not significant**
- How you would debug, refine metrics, and decide next steps.
---
## Prompt 2 — Business case: volatility up, retail trading volume down
Recently **BTC volatility increased significantly**, but **retail user trading volume decreased**.
Describe how you would investigate this end-to-end:
- How you would validate the data and define the metric(s).
- How you would decompose the drop (e.g., users × frequency × size) and segment the problem.
- What product, market-structure, and risk/compliance factors could explain the pattern.
- What analyses or follow-up experiments you would run, and what decisions you could support.
Quick Answer: This question evaluates a data scientist's competency in experiment design, metric specification and validation, causal inference, diagnostic decomposition of volume changes, segmentation, and product- and risk-oriented analytics.