Launch and measure a TV campaign
Overview
This question evaluates a data scientist's competency in experimental design and causal inference, including geo-level matching, KPI definition, media measurement (GRP/TRP, reach/frequency, adstock and saturation modeling), power analysis, and operational risk management for measuring incremental vaccinations.
CVS Health
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0
6-Week Linear TV Experiment to Increase Flu Vaccinations
Design a 6-week linear TV campaign and its measurement plan to causally estimate incremental flu vaccinations. Assume access to DMA-level verified vaccinations, media delivery (GRP/TRP), and basic operational data (inventory, staffing).
Scope
- Select 12 test DMAs from the 210 U.S. DMAs and assign matched controls (1:1).
- Define the KPI, causal identification strategy, media plan (TRPs, dayparts, reach/frequency), and modeling choices (adstock, saturation).
- Address execution risks (spillover, concurrent media, shocks, supply/ops constraints) and outline power analysis and triangulation.
Requirements
-
DMA Selection and Matching
- Choose 12 test DMAs and 12 matched controls.
- Describe matching criteria and method (e.g., distance metric, pair matching/stratification), pre-period length used for matching, and exclusion rules.
- Explain how you will avoid or control for news/sports/holiday shocks in market selection and scheduling.
-
KPI and Causal Design
- Primary KPI: incremental verified vaccinations per DMA over the 6-week post period (and per-capita normalization).
- Choose and justify a causal design: geo-randomized experiment with matched pairs (preferred), difference-in-differences, and/or synthetic controls for sensitivity.
- Specify how you will handle market spillovers, unequal TRP delivery, and concurrent media.
-
Media Plan Parameters
- GRP/TRP targets by demo, weekly distribution, and total.
- Daypart mix and content exclusions to mitigate shocks.
- Reach-frequency goals and how you will estimate/verify them.
- Modeling of adstock/decay and saturation (include formulas/assumptions).
-
Measurement and Analysis
- Pre-period length and cadence; checks for parallel trends.
- Estimation approach (e.g., DiD regression), weighting, and covariates.
- Power and sample size: show how you’d compute Minimum Detectable Effect (MDE) using market-level variance; include a worked numeric example.
- Guardrails (e.g., call-center load, pharmacy stockouts) and pause criteria.
- Triangulation with MMM and pharmacy footfall data; how to reconcile findings.
Solution
Login required
Show