Estimate Successful Sign-ups from Super Bowl QR Code Ad

Quick Overview

Evaluates measurement of successful sign-ups from a Super Bowl QR-code advertisement. Strong answers combine top-down funnel modeling, bottom-up log measurement, incrementality analysis, deduplication, and validation.

Estimate Successful Sign-ups from Super Bowl QR Code Ad

Company: Coinbase

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

##### Scenario A cryptocurrency company airs a Super Bowl commercial featuring a QR code; viewers who scan and complete sign-up receive a $15 coupon. ##### Question How would you estimate the total number of successful sign-ups generated by this Super Bowl advertisement? ##### Hints Consider audience size, scan rate, funnel conversion, historical comps, coupon redemption, and ways to validate assumptions with sampling or controlled experiments.

Overview: Evaluates measurement of successful sign-ups from a Super Bowl QR-code advertisement. Strong answers combine top-down funnel modeling, bottom-up log measurement, incrementality analysis, deduplication, and validation.

|Home/Analytics & Experimentation/Coinbase
Coinbase logo
Coinbase
Jul 12, 2025
mediumData ScientistTechnical ScreenAnalytics & Experimentation
22
0

Estimate Successful Sign-ups from a Super Bowl QR Code Ad

A cryptocurrency company airs a national Super Bowl commercial featuring a bouncing QR code. Viewers who scan and complete sign-up receive a $15 coupon.

Assume a successful sign-up means a new, verified account, including KYC completion, within a 7-day attribution window after ad exposure. Estimate total successful sign-ups generated by the advertisement.

Constraints & Assumptions

  • Distinguish attributed sign-ups from incremental sign-ups.
  • Deduplicate users across QR scans, landing pages, app installs, devices, and accounts.
  • Include uncertainty around exposure, scan rate, conversion, attribution, and KYC completion.
  • Treat organic background activity and other marketing campaigns as potential confounders.

Clarifying Questions to Ask Guidance

  • What were the exact air times, markets, audience estimates, and streaming reach?
  • Was the QR URL unique to the ad and timestamped in logs?
  • Were there other promotions, press events, outages, or referral campaigns during the window?
  • Is the business asking for gross attributed sign-ups or incremental lift?

Part 1 - Define Metric and Attribution Window

Define the outcome, attribution rules, and deduplication approach.

What This Part Should Cover Guidance

  • Count unique new verified accounts that complete KYC within the chosen window.
  • Define scan, landing, install, account creation, KYC completion, coupon issuance, and redemption events.
  • Deduplicate by account, device, hashed identifiers, and fraud controls.
  • Explain primary and sensitivity windows such as 7 days and longer-tail checks.

Part 2 - Top-Down Funnel

Build a top-down estimate from audience exposure to verified sign-ups.

What This Part Should Cover Guidance

  • Start with exposed viewers and apply rates for attention, scan, landing conversion, install, account creation, and KYC completion.
  • Use ranges or distributions rather than a single false-precision estimate.
  • Calibrate assumptions with historical campaigns or industry benchmarks if available.
  • Propagate uncertainty through the funnel.

Part 3 - Bottom-Up Measurement

Use available logs and business data to measure attributed sign-ups directly.

What This Part Should Cover Guidance

  • Use QR URL logs, landing-page events, app-store install data, referral or coupon codes, account creation, and KYC logs.
  • Build a timestamped funnel and identify drop-off points.
  • Handle cross-device flows and delayed KYC completion.
  • Reconcile coupon issuance and redemption with verified-account records.

Part 4 - Incrementality and Validation

Explain how to separate incremental lift from background activity and validate assumptions.

What This Part Should Cover Guidance

  • Compare against pre-period baselines, synthetic controls, geo or time controls, and expected organic trends.
  • Account for other marketing, media coverage, crypto-market movement, and site outages.
  • Validate with survey recall, holdout markets if possible, search trends, or controlled follow-up campaigns.
  • Report attributed sign-ups, incremental sign-ups, confidence intervals, and caveats separately.

Follow-up Questions Guidance

  • What if the QR landing page crashed during the ad?
  • How would you estimate sign-ups if users scanned on TV but finished on mobile later?
  • How would you value the campaign if sign-ups are high but funded-account conversion is low?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Coinbase•More Data Scientist•Coinbase Data Scientist•Coinbase Analytics & Experimentation•Data Scientist Analytics & Experimentation