Design incrementality test for TikTok ads
Company: TikTok
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Onsite
From the advertiser perspective, design an incrementality test to prove TikTok ads drive lift in conversions. Specify: (1) the unit of randomization (user, cookie, geo, advertiser) and justification; (2) holdout construction (ghost ads/PSA, pre-bid randomization, or geo holdout) and contamination controls; (3) primary KPI (incremental conversions and iROAS) and exact formulas; (4) sample size and duration assumptions given baseline conversion rate and expected lift; (5) how to handle cross-channel spillovers and auction dynamics; (6) guardrails (organic traffic diversion, quality) and falsification tests (AA, placebo periods); and (7) how you’d report heterogeneous lift by audience and creative while avoiding p-hacking.
Quick Answer: This question evaluates a data scientist's mastery of causal inference, experiment design, and measurement of advertising incrementality within the analytics and experimentation domain, requiring both practical application skills and conceptual understanding.