Design incrementality test for TikTok ads
Overview
This question evaluates a data scientist's mastery of causal inference, experiment design, and measurement of advertising incrementality within the analytics and experimentation domain, requiring both practical application skills and conceptual understanding.
TikTok
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Analytics & Experimentation
0
0
Design an Incrementality Test to Prove TikTok Ads Drive Lift in Conversions
You are an advertiser who wants to causally prove that TikTok ads increase conversions (and revenue). Design a rigorous incrementality experiment that covers the following:
- Unit of randomization and justification (e.g., user, cookie, geo, advertiser).
- Holdout construction (ghost ads/PSA, pre-bid randomization, or geo holdout) and contamination controls.
- Primary KPIs and exact formulas for incremental conversions and iROAS.
- Sample size and duration assumptions given a baseline conversion rate and expected lift.
- How to handle cross-channel spillovers and auction dynamics.
- Guardrails (organic traffic diversion, quality) and falsification tests (AA test, placebo periods).
- How to report heterogeneous lift by audience and creative while avoiding p-hacking.
Solution
Login required
Show