Design a Controlled TV-Advertising Experiment for Sign-Ups

Q: Design a Controlled TV-Advertising Experiment for Sign-Ups

This question evaluates a data scientist's skills in experimental design, causal impact measurement for marketing channels, statistical power and sample-size calculations, KPI definition, and dealing with temporal and spatial confounders within the Analytics & Experimentation domain.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Controlled TV Advertising Experiment: Incremental Website Sign-ups

Context

You are asked to design a controlled experiment to quantify the incremental impact of TV ads on website sign-ups. TV here includes linear TV and/or CTV that can be purchased at a geographic level (e.g., DMA). You have historical web sign-ups by geo and time and can avoid national TV during the test window.

Task

Design the experiment and specify:

Hypothesis (H0/H1).
Randomization method and experimental unit (e.g., geo holdouts) and how you’ll select/stratify markets.
Rollout design and timeline (including time-based ON/OFF if used) and how you’ll handle carryover/adstock.
KPIs (primary, secondary, guardrails) and how you’ll measure lift.
Required sample size, MDE, power, and test length; show how you’d compute it and provide a concrete numeric example.
Success criteria (statistical and business/ROI).
Key risks and mitigations (e.g., spillover across geos, concurrent channels, seasonality).

Hint: Consider geo holdouts, time-based rollout, and lift measurement.

Design a Controlled TV-Advertising Experiment for Sign-Ups

Controlled TV Advertising Experiment: Incremental Website Sign-ups

Context

Task

Solution

Comments (0)