Design KYC experiment amid crypto volatility
Company: Coinbase
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
You're the analyst for a crypto exchange’s mobile onboarding funnel. KYC completion rate dropped 8% WoW during a period of high BTC volatility. Design an A/B test to improve completion without increasing fraud. Specify: a) unit of randomization (user/session/geo) and why, b) primary metric and at least two guardrails (e.g., fraud approval rate, time-to-approve, support tickets per 1k signups), c) how to neutralize exogenous market shocks (e.g., include BTC/ETH price and realized volatility as covariates with CUPED or run a Difference‑in‑Differences against a stable geo holdout), d) sample size, expected duration, and a sequential monitoring plan that controls Type I error (e.g., alpha‑spending or group‑sequential boundaries), e) country‑specific compliance step heterogeneity and app store review delays, and f) a precise pre‑registration (hypotheses, data exclusions, stopping rules, analysis plan).
Quick Answer: The prompt evaluates proficiency in experiment design and causal inference under external market shocks, covering competencies in metric definition and guardrails, sample size and sequential monitoring planning, and managing compliance and operational heterogeneity for an Analytics & Experimentation Data Scientist role.