PracHub
QuestionsLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Meta

Diagnose a sudden KPI drop and validate causes

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's skills in root-cause analysis, metric instrumentation verification, segment-wise localization, attribution of internal versus external causes, and time-series causal-impact and anomaly-detection methods.

  • medium
  • Meta
  • Analytics & Experimentation
  • Data Scientist

Diagnose a sudden KPI drop and validate causes

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Onsite

A core KPI (comments_per_DAU) suddenly drops materially. Outline a structured root-cause analysis and validation plan. a) Scoping and sanity: Quantify the drop (absolute and relative), confirm it across dashboards and raw logs, and rule out instrumentation (release notes, schema changes, event loss, clock skew). What guardrail metrics do you check first and why? b) Slice-and-localize: Describe the exact segmentation you would run (platform/app version, geo, cohort, time-of-day, traffic source, user tenure, content category, creators vs consumers) and the funnel steps to inspect. How do you separate mix-shift vs within-segment degradation? c) Attribution: Enumerate internal and external potential causes (code deploys/feature flags, AB tests, recommender changes, moderation policies, outages; holidays/news, competitor actions). For each, propose a validation: quick reversion, killing a flag, switchback on suspect features, holdout comparisons, difference-in-differences with an unaffected market, placebo tests. d) Time-series methods: Specify an anomaly detection or causal impact approach (e.g., STL + CUSUM, BSTS/SCUL) with controls. What prior period and controls would you pick, and how do you guard against post-treatment contamination? e) Decision and follow-up: Define thresholds for declaring a root cause vs correlation, the rollback/mitigation plan, and how you’d monitor recovery and prevent recurrence (alerts, canaries, auto rollbacks, dashboards).

Quick Answer: This question evaluates a data scientist's skills in root-cause analysis, metric instrumentation verification, segment-wise localization, attribution of internal versus external causes, and time-series causal-impact and anomaly-detection methods.

Related Interview Questions

  • Evaluate a Live-Stream Group Notification Under Network Effects - Meta (medium)
  • Define Success for a New Group Feature Without Hiding Cannibalization - Meta (medium)
  • Compare Shop and Web Ad Performance Without Overclaiming - Meta (medium)
  • Evaluate a New Ads-Ranking Algorithm - Meta (medium)
  • Measure scheduled posts feature success - Meta (medium)
|Home/Analytics & Experimentation/Meta

Diagnose a sudden KPI drop and validate causes

Meta logo
Meta
Oct 13, 2025, 9:49 PM
mediumData ScientistOnsiteAnalytics & Experimentation
2
0

A core KPI (comments_per_DAU) suddenly drops materially. Outline a structured root-cause analysis and validation plan.

a) Scoping and sanity: Quantify the drop (absolute and relative), confirm it across dashboards and raw logs, and rule out instrumentation (release notes, schema changes, event loss, clock skew). What guardrail metrics do you check first and why?

b) Slice-and-localize: Describe the exact segmentation you would run (platform/app version, geo, cohort, time-of-day, traffic source, user tenure, content category, creators vs consumers) and the funnel steps to inspect. How do you separate mix-shift vs within-segment degradation?

c) Attribution: Enumerate internal and external potential causes (code deploys/feature flags, AB tests, recommender changes, moderation policies, outages; holidays/news, competitor actions). For each, propose a validation: quick reversion, killing a flag, switchback on suspect features, holdout comparisons, difference-in-differences with an unaffected market, placebo tests.

d) Time-series methods: Specify an anomaly detection or causal impact approach (e.g., STL + CUSUM, BSTS/SCUL) with controls. What prior period and controls would you pick, and how do you guard against post-treatment contamination?

e) Decision and follow-up: Define thresholds for declaring a root cause vs correlation, the rollback/mitigation plan, and how you’d monitor recovery and prevent recurrence (alerts, canaries, auto rollbacks, dashboards).

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation

Write your answer

Your first approved answer each day earns 20 XP.

Sign in to write your answer.
PracHub

Master your tech interviews with 9,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • AI Coding Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.