Investigate Super Bowl Ad Impact on User Sign-Ups and Revenue
Company: Coinbase
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
A crypto exchange airs a Super Bowl commercial featuring a QR code that awards a $15 coupon to anyone who scans it and completes sign-up. Shortly afterward, the PM reports that retail revenue has fallen 20% and asks for analysis and next steps.
##### Question
1. **Estimate the sign-ups from the QR-code ad.** How would you estimate the number of users who will complete sign-up after scanning the QR code shown during the Super Bowl ad? Walk through the funnel (reach → scans → sign-ups), and justify your CTR/CVR (scan-rate and conversion-rate) assumptions with benchmarks. Optionally, extend the estimate to KYC-complete and funded users.
2. **Validate the estimate with measurement.** How would you instrument and measure the actual incremental sign-ups, rather than relying only on a forward model?
3. **Investigate the 20% retail revenue decline.** How would you investigate the drivers of the drop, what analyses would you run next, and how would you decompose revenue to locate the cause?
4. **Use cohort analysis.** How would different user cohorts — especially the Super Bowl QR cohort — factor into your analysis, and how would they shape your recommendations?
5. **Recommend fixes and experiments.** Given likely root causes, what mitigations and experiments would you propose?
##### Hints
Build a reach → scan → sign-up funnel and sanity-check the rates against TV-QR benchmarks. For the revenue drop, decompose the metric (e.g., active traders × trades/user × avg trade size × take rate), segment by dimensions, control for crypto market factors, run cohort and retention analysis, then suggest mitigations or experiments.
Quick Answer: Coinbase data scientist case on estimating Super Bowl QR-code ad sign-ups, measuring incrementality, decomposing a retail revenue decline, using cohort analysis, and proposing experiments or mitigations.