Evaluate Key Metrics for Capital One Ad Campaign

Quick Overview

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate Key Metrics for Capital One Ad Campaign states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Evaluate Key Metrics for Capital One Ad Campaign

Company: Capital One

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

##### Scenario Capital One launches the Quicksilver credit-card marketing campaign on a streaming platform that can run either a 30-second unskippable ad or a skippable ad. ##### Question Before placing any ads, what key factors (metrics, customer segments, risks) would you evaluate for this campaign? Using 100 000 viewers, a fixed cost of $50 000, $0.10 cost per view, $500 revenue per approved account, a 1.4 % click-to-apply conversion, and a 50 % approval rate, calculate the campaign’s profit or loss. If we switch to a skippable ad that costs $0.30 only when watched and 80 % of viewers skip, what minimum conversion rate keeps profit equal to the first plan? Suppose a third option is to run no campaign at all. How would you decide which of the three options to choose? With extra time, what additional data or analyses would you request to strengthen your recommendation? ##### Hints Lay out revenue-cost equation, compute profit, then set profit_scenario2 = profit_scenario1 to solve for required conversion; discuss CAC, LTV, incremental lift, brand impact, uncertainty.

Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate Key Metrics for Capital One Ad Campaign states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Analytics & Experimentation/Capital One
Capital One logo
Capital One
Aug 4, 2025, 10:55 AM
mediumData ScientistTechnical ScreenAnalytics & Experimentation
7
0

Evaluate Key Metrics for Capital One Ad Campaign

Streaming Ad Campaign Evaluation and ROI Comparison

Context

You are evaluating a credit-card marketing campaign on a streaming platform. The platform can run either a 30-second unskippable ad or a skippable ad. You must identify what to evaluate before launching, compute the profit for the unskippable plan, find the conversion needed for the skippable plan to match that profit, compare against the option of running no campaign, and propose additional data/analyses to strengthen a recommendation.

Assumption for clarity: The 1.4% "click-to-apply conversion" is treated as the overall view-to-application rate (i.e., fraction of all viewers who submit an application). Approval rate is the fraction of applications approved.

Given

  • Viewers (impressions delivered): 100,000
  • Fixed cost (creative/production): $50,000
  • Unskippable ad: cost per view (CPV) = $0.10
  • Revenue per approved account: $500
  • View-to-application rate: 1.4%
  • Approval rate: 50%

Skippable variant:

  • Cost per watched view: $0.30
  • 80% skip rate (so 20% watch)

Tasks

  1. Before placing any ads, list key factors to evaluate (metrics, customer segments, risks).
  2. Compute profit or loss for the unskippable campaign with the given inputs.
  3. For the skippable ad, what minimum application conversion rate (among watchers) keeps profit equal to the unskippable plan? Also report the implied overall application rate across all 100,000 viewers.
  4. With a third option of no campaign, how would you decide among the three options?
  5. With extra time, what additional data or analyses would you request to strengthen your recommendation?

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
  • State assumptions about instrumentation, randomization, sample size, and data quality.
  • Separate descriptive analysis from causal claims.

What a Strong Answer Covers Guidance

  • A metric framework with primary, guardrail, and diagnostic metrics.
  • A credible analysis or experiment design with clear assumptions and bias checks.
  • SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
  • An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions Guidance

  • What sanity checks would you run before trusting the result?
  • How would you handle novelty effects, seasonality, or selection bias?
  • What decision would you make if metrics disagree?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Capital One•More Data Scientist•Capital One Data Scientist•Capital One Analytics & Experimentation•Data Scientist Analytics & Experimentation