Choosing Metrics to Judge Whether a Video Ad Works: Reach, Attention and Lift
Company: Rakuten
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
As a product manager working on video advertising, explain which key metrics you would use to judge whether a video ad is effective. For each metric, say what it tells you and how it is calculated, explain how the right set changes with what the advertiser wants to achieve, and say which metric you would treat as primary.
```hint Let the goal pick the metric
A campaign meant to build awareness and one meant to drive purchases judge the same video very differently. Arrange the metrics by stage, from the ad being shown and actually seen to it changing what people do, and ask which stages need a comparison group before you can claim the ad caused anything.
```
### Clarifying Questions
- Is the campaign aiming at brand awareness and consideration, or at a direct response such as app installs, sign-ups or purchases?
- Which formats are in scope: skippable ads before streamed content, non-skippable ads, ads that autoplay in a feed or article, or ads on connected TVs? What counts as a "view" depends on this.
- Are we measuring from the advertiser's side (did this campaign work) or as the owner of the video ad product (is the product delivering value to advertisers and to viewers)?
- What measurement is available: click and conversion tracking, brand-lift surveys, or randomized holdout groups?
### What a Strong Answer Covers
- Choosing metrics from the campaign goal and the ad format rather than reciting one universal list.
- Precise definitions of delivery, viewability, reach and frequency, and attention metrics such as view rate, quartile completion and watch time.
- Outcome and efficiency measures: brand lift, conversions, cost per completed view, cost per acquisition and return on ad spend.
- Proving causation with incrementality tests instead of trusting last-touch or view-through attribution.
- Data-quality and viewer-experience guardrails, and the difference between a metric and a dimension used to segment it.
### Follow-up Questions
- A campaign shows a high completion rate, but sales did not move. How would you find out why?
- How would you design a test that measures the incremental lift of a video campaign?
- If you owned the video ad product, which metric would you track to make sure ads are not driving viewers away from the content?
Overview: A product analytics question asking which metrics show whether a video ad is working. It tests tying metrics to the campaign goal and ad format, defining delivery, viewability, attention and outcome measures precisely, and proving incremental impact with experiments rather than trusting attribution.