Diagnose a 20% retail revenue drop
Company: Coinbase
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
Your e-commerce retailer reports weekly revenue fell 20% in Week T versus the 4-week moving average baseline. Build a rigorous diagnosis and quantify each driver’s contribution.
Known context and signals:
- Traffic sessions were flat (+0.5% WoW), but conversion rate (CVR) fell and average order value (AOV) changed.
- Returns rose (7% → 10%), cancellations (1% → 2%), stockouts on top-50 SKUs increased (3% → 9%).
- A price increase (+8%) started 3 weeks prior; a new $75 free-shipping threshold began 2 weeks prior; paid spend mix shifted toward upper-funnel; a competitor ran a 25%-off sitewide sale.
Tasks:
1) Decompose revenue delta into traffic × CVR × AOV, and further into mix/price/discount effects, post-purchase deductions (returns/cancellations), and stockout lost sales. Provide formulas and a stepwise waterfall showing approximate percentage-point contribution of each factor.
2) Identify the quickest falsifiable checks you’d run within 24 hours (e.g., device-specific CVR drop, OOS by category, promo overlap, page latency, checkout errors, funnel step breakage, acquisition channel mix, new vs returning cohorts).
3) Causal assessment: propose a geo- or cohort-level DiD or interruption model to isolate the free-shipping change from the competitor sale and seasonality. Specify control groups, pre-trends checks, and robustness tests.
4) Experiments and fixes: outline targeted tests (pricing, threshold A/B, back-in-stock alerts, assortment substitution) and the success metrics you’d monitor (lift, profit, LTV impact).
Quick Answer: This question evaluates a data scientist's ability in revenue decomposition, diagnostic analytics, causal inference, and experimentation design, covering traffic×CVR×AOV breakdowns, post-purchase deductions, stockout impacts, rapid falsifiable checks, and difference-in-differences-style attribution.