PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Analytics & Experimentation/TikTok

Diagnose metric drop in Ads Manager

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a Data Scientist's competency in operational analytics, metric integrity, root-cause analysis, and experiment interpretation, covering concepts such as logging latency, sample ratio assessment, and product instrumentation awareness.

  • hard
  • TikTok
  • Analytics & Experimentation
  • Data Scientist

Diagnose metric drop in Ads Manager

Company: TikTok

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

Your Ads Manager dashboard shows a 15% relative drop in ad-creation completion rate on 2025-06-10 versus its prior 7-day baseline. The metric is defined as completed_creations / initiated_creations per day at the advertiser level. Describe a step-by-step triage and deep-dive plan to: (1) verify the drop (data quality checks, AA tests, sample ratio mismatch, logging latency); (2) localize the issue (slice by geo, platform, entry point, advertiser tenure/spend tier, funnel step, experiment flags, recent UI/BE releases); (3) determine root cause (product change vs. traffic mix vs. seasonality vs. incidents) using specific analyses you would run; (4) quantify impact and propose immediate mitigations; and (5) define decision criteria for rollback vs. continue. Be specific about queries/cuts you would run and guardrail metrics you would check.

Quick Answer: This question evaluates a Data Scientist's competency in operational analytics, metric integrity, root-cause analysis, and experiment interpretation, covering concepts such as logging latency, sample ratio assessment, and product instrumentation awareness.

Related Interview Questions

  • Define Ultra success metrics and detect suspicious transactions - TikTok (easy)
  • Plan DS approach for biker delivery project - TikTok (easy)
  • Define and critique a user activity metric - TikTok (easy)
  • Design and decompose Trust & Safety risk metrics - TikTok (easy)
  • Analyze promo anomaly and design risk guardrails - TikTok (Medium)
TikTok logo
TikTok
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Analytics & Experimentation
5
0

Investigate a 15% Drop in Ad-Creation Completion Rate

Context

On 2025-06-10, your Ads Manager dashboard shows a 15% relative decrease in the ad-creation completion rate versus the prior 7-day baseline. The metric is defined daily at the advertiser level as:

  • completion rate = completed_creations / initiated_creations

Assume the KPI is reported for all UI-based ad-creation flows (not API) and that daily processing may have up to a few hours of logging latency.

Task

Describe a step-by-step triage and deep-dive plan to:

  1. Verify the drop (data quality checks, A/A tests, sample ratio mismatch, logging latency).
  2. Localize the issue (slice by geo, platform, entry point, advertiser tenure/spend tier, funnel step, experiment flags, recent UI/BE releases).
  3. Determine root cause (product change vs. traffic mix vs. seasonality vs. incidents) using specific analyses.
  4. Quantify impact and propose immediate mitigations.
  5. Define decision criteria for rollback vs. continue.

Be specific about the queries/cuts you would run and the guardrail metrics you would check.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More TikTok•More Data Scientist•TikTok Data Scientist•TikTok Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.