PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Analytics & Experimentation/TikTok

Prove conversion ads value via incrementality

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a candidate's competency in causal inference and experimentation design for advertising measurement, covering metrics and attribution, statistical power and sample-size calculations, interference handling, and business-level impact estimation.

  • hard
  • TikTok
  • Analytics & Experimentation
  • Data Scientist

Prove conversion ads value via incrementality

Company: TikTok

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

An e‑commerce advertiser runs conversion‑optimized ads and wants proof the ads cause incremental conversions (lift). Propose a rigorous plan: (1) choose among user‑level RCT holdouts, geo‑experiments with matched markets, or PSA/ghost‑bid approaches; justify assumptions and spillover risks; (2) define the primary metric (absolute/relative lift in conversion rate) and attribution windows; address view‑through bias, cross‑device, and post‑view latency; (3) perform sample‑size/power calculations to detect a 2% relative lift from a 3% baseline CR at α=0.05, stating formulas and assumptions; (4) outline analysis (difference‑in‑differences, CUPED, pre‑trend diagnostics), interference handling, budget/pacing constraints, and how to guard against p‑hacking; (5) compute incremental ROAS with confidence intervals and present results/actionability to the advertiser, including sensitivity analyses and repeatability criteria.

Quick Answer: This question evaluates a candidate's competency in causal inference and experimentation design for advertising measurement, covering metrics and attribution, statistical power and sample-size calculations, interference handling, and business-level impact estimation.

Related Interview Questions

  • Define Ultra success metrics and detect suspicious transactions - TikTok (easy)
  • Plan DS approach for biker delivery project - TikTok (easy)
  • Define and critique a user activity metric - TikTok (easy)
  • Design and decompose Trust & Safety risk metrics - TikTok (easy)
  • Analyze promo anomaly and design risk guardrails - TikTok (Medium)
TikTok logo
TikTok
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
1
0
Loading...

Measuring Incremental Lift of Conversion-Optimized Ads

Context

An e-commerce advertiser is running conversion-optimized ads and requests rigorous proof that the ads cause incremental conversions (lift). You are asked to design and analyze a trustworthy measurement plan that isolates causality under real-world constraints (auctions, pacing, cross-device, view-throughs).

Task

Propose a defensible, end-to-end plan that covers:

  1. Design choice and assumptions
    • Choose among: user-level RCT holdouts, geo-experiments with matched markets, or PSA/ghost-bid approaches.
    • Justify the choice, clearly state assumptions, and discuss spillover/interference risks.
  2. Metrics and attribution windows
    • Define the primary metric (absolute and/or relative lift in conversion rate) and the attribution windows.
    • Address view-through bias, cross-device identity, and post-view latency.
  3. Power and sample size
    • Perform sample-size/power calculations to detect a 2% relative lift from a 3% baseline conversion rate (CR) at α = 0.05.
    • Show formulas, parameters, and assumptions.
  4. Analysis plan and guardrails
    • Outline the analysis (e.g., difference-in-differences, CUPED, pre-trend diagnostics), interference handling, budget/pacing constraints, and how to guard against p-hacking.
  5. Business readout
    • Compute incremental ROAS with confidence intervals.
    • Explain how to present results and actionability to the advertiser, including sensitivity analyses and repeatability criteria.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More TikTok•More Data Scientist•TikTok Data Scientist•TikTok Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.