Design incrementality test for TikTok ads

Quick Overview

This question evaluates a data scientist's mastery of causal inference, experiment design, and measurement of advertising incrementality within the analytics and experimentation domain, requiring both practical application skills and conceptual understanding.

Design incrementality test for TikTok ads

Company: TikTok

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

From the advertiser perspective, design an incrementality test to prove TikTok ads drive lift in conversions. Specify: (1) the unit of randomization (user, cookie, geo, advertiser) and justification; (2) holdout construction (ghost ads/PSA, pre-bid randomization, or geo holdout) and contamination controls; (3) primary KPI (incremental conversions and iROAS) and exact formulas; (4) sample size and duration assumptions given baseline conversion rate and expected lift; (5) how to handle cross-channel spillovers and auction dynamics; (6) guardrails (organic traffic diversion, quality) and falsification tests (AA, placebo periods); and (7) how you’d report heterogeneous lift by audience and creative while avoiding p-hacking.

Quick Answer: This question evaluates a data scientist's mastery of causal inference, experiment design, and measurement of advertising incrementality within the analytics and experimentation domain, requiring both practical application skills and conceptual understanding.

|Home/Analytics & Experimentation/TikTok
TikTok logo
TikTok
Oct 13, 2025, 9:49 PM
hardData ScientistOnsiteAnalytics & Experimentation
6
0

Design an Incrementality Test to Prove TikTok Ads Drive Lift in Conversions

You are an advertiser who wants to causally prove that TikTok ads increase conversions (and revenue). Design a rigorous incrementality experiment that covers the following:

  1. Unit of randomization and justification (e.g., user, cookie, geo, advertiser).
  2. Holdout construction (ghost ads/PSA, pre-bid randomization, or geo holdout) and contamination controls.
  3. Primary KPIs and exact formulas for incremental conversions and iROAS.
  4. Sample size and duration assumptions given a baseline conversion rate and expected lift.
  5. How to handle cross-channel spillovers and auction dynamics.
  6. Guardrails (organic traffic diversion, quality) and falsification tests (AA test, placebo periods).
  7. How to report heterogeneous lift by audience and creative while avoiding p-hacking.
Loading comments...