Investigate Super Bowl Ad Impact on User Sign-Ups and Revenue

Quick Overview

Coinbase data scientist case on estimating Super Bowl QR-code ad sign-ups, measuring incrementality, decomposing a retail revenue decline, using cohort analysis, and proposing experiments or mitigations.

Investigate Super Bowl Ad Impact on User Sign-Ups and Revenue

Company: Coinbase

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

##### Scenario A crypto exchange airs a Super Bowl commercial featuring a QR code that awards a $15 coupon to anyone who scans it and completes sign-up. Shortly afterward, the PM reports that retail revenue has fallen 20% and asks for analysis and next steps. ##### Question 1. **Estimate the sign-ups from the QR-code ad.** How would you estimate the number of users who will complete sign-up after scanning the QR code shown during the Super Bowl ad? Walk through the funnel (reach → scans → sign-ups), and justify your CTR/CVR (scan-rate and conversion-rate) assumptions with benchmarks. Optionally, extend the estimate to KYC-complete and funded users. 2. **Validate the estimate with measurement.** How would you instrument and measure the actual incremental sign-ups, rather than relying only on a forward model? 3. **Investigate the 20% retail revenue decline.** How would you investigate the drivers of the drop, what analyses would you run next, and how would you decompose revenue to locate the cause? 4. **Use cohort analysis.** How would different user cohorts — especially the Super Bowl QR cohort — factor into your analysis, and how would they shape your recommendations? 5. **Recommend fixes and experiments.** Given likely root causes, what mitigations and experiments would you propose? ##### Hints Build a reach → scan → sign-up funnel and sanity-check the rates against TV-QR benchmarks. For the revenue drop, decompose the metric (e.g., active traders × trades/user × avg trade size × take rate), segment by dimensions, control for crypto market factors, run cohort and retention analysis, then suggest mitigations or experiments.

Quick Answer: Coinbase data scientist case on estimating Super Bowl QR-code ad sign-ups, measuring incrementality, decomposing a retail revenue decline, using cohort analysis, and proposing experiments or mitigations.

|Home/Analytics & Experimentation/Coinbase
Coinbase logo
Coinbase
Jul 12, 2025, 6:59 PM
mediumData ScientistTechnical ScreenAnalytics & Experimentation
9
0

Coinbase Super Bowl QR Ad: Sign-Ups, Revenue Impact, and Next Steps

A crypto exchange airs a Super Bowl commercial featuring a QR code that awards a $15 coupon to anyone who scans it and completes sign-up. Shortly afterward, the PM reports that retail revenue has fallen 20% and asks for analysis and next steps.

Constraints & Assumptions

  • Treat this as an analytics and experimentation case, not only a market-sizing question.
  • You may use reasonable funnel assumptions, but state ranges and uncertainty.
  • Separate sign-up volume from incremental high-quality users, funded users, and revenue impact.
  • Assume crypto-market conditions, seasonality, promotions, product outages, and cohort mix can all affect revenue.

Clarifying Questions to Ask Guidance

  • What does "retail revenue" include: trading fees, spread, subscription, staking, or other revenue?
  • Is the 20% decline week-over-week, year-over-year, or versus forecast?
  • Did the coupon apply only to new users, and did it require KYC or funding?
  • Were there outages, pricing changes, fee changes, market movements, or campaign overlaps around the ad?
  • What tracking exists for QR scans, landings, sign-ups, KYC, funding, and trades?

What a Strong Answer Covers Guidance

  • A forward funnel estimate: reach, scan rate, unique landing rate, sign-up conversion, KYC completion, funding, and first-trade conversion.
  • Instrumentation and causal measurement for actual incremental sign-ups, such as tagged URLs, holdouts, geo/time comparisons, incrementality checks, and deduplication.
  • Revenue decomposition: active traders, trades per trader, average trade size, take rate, product mix, asset mix, promotions, and refunds or credits.
  • Cohort analysis separating the Super Bowl cohort from existing users and other acquisition cohorts.
  • Segmentation by new versus existing, funded versus unfunded, KYC status, geography, device, traffic source, and risk/fraud quality.
  • Plausible root causes for a revenue decline, including market movement, coupon dilution, low-quality users, cannibalization, conversion friction, outages, or fee/product changes.
  • Recommended mitigations and experiments with primary metrics and guardrails.

Follow-up Questions Guidance

  • How would you estimate incrementality if everyone saw the national ad?
  • What would make the campaign successful even if short-term revenue fell?
  • How would you detect fraud or coupon abuse?
  • Which dashboard would you build for the PM in the first 24 hours?
Loading comments...