Design incrementality test for TikTok ads
Company: TikTok
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Onsite
From the advertiser perspective, design an incrementality test to prove TikTok ads drive lift in conversions. Specify: (1) the unit of randomization (user, cookie, geo, advertiser) and justification; (2) holdout construction (ghost ads/PSA, pre-bid randomization, or geo holdout) and contamination controls; (3) primary KPI (incremental conversions and iROAS) and exact formulas; (4) sample size and duration assumptions given baseline conversion rate and expected lift; (5) how to handle cross-channel spillovers and auction dynamics; (6) guardrails (organic traffic diversion, quality) and falsification tests (AA, placebo periods); and (7) how you’d report heterogeneous lift by audience and creative while avoiding p-hacking.
Overview: This question evaluates a data scientist's mastery of causal inference, experiment design, and measurement of advertising incrementality within the analytics and experimentation domain, requiring both practical application skills and conceptual understanding.
Read the full TikTok Data Scientist interview experience this question came from