PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Meta

Diagnose sudden KPI drop with segmentation

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's skills in production incident diagnostics, instrumentation and data-validation checks, segmented funnel and localization analysis, falsifiable hypothesis formulation, and rapid root-cause decision-making under time pressure.

  • hard
  • Meta
  • Analytics & Experimentation
  • Data Scientist

Diagnose sudden KPI drop with segmentation

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

On 2025-09-01, daily Likes actions on a global social network fell 10% versus the prior 14-day average, while DAU stayed flat. Treat this as a production incident. Outline an end-to-end diagnostic plan: 1) enumerate concrete data-validation checks you will run (e.g., event volume parity vs app_sessions, null-rate and schema-change diffs, client→server receipt ratios) and what thresholds scream instrumentation failure; 2) rule out expected variability by checking seasonality, regional holidays, release calendars, and major outages/events—what external data would you join and how; 3) localize the drop by funnel stage (feed impressions → post views → like impressions → likes) and by segments (surface: friends vs pages vs events; platform; app version; geography; new vs retained users); 4) state three falsifiable hypotheses and the exact cuts/queries/plots to test each; 5) define the decision tree that gets you to a root cause within 24 hours, including what you’d do if multiple segments move in opposite directions; 6) propose one immediate mitigation and one follow-up experiment to prevent recurrence. List the first three queries or charts you would run and the precise outcomes that would make you conclude behavior change (not logging) is at fault.

Quick Answer: This question evaluates a data scientist's skills in production incident diagnostics, instrumentation and data-validation checks, segmented funnel and localization analysis, falsifiable hypothesis formulation, and rapid root-cause decision-making under time pressure.

Related Interview Questions

  • Measure scheduled posts feature success - Meta (medium)
  • Estimate ads ranking revenue impact - Meta (medium)
  • How should you evaluate unconnected content? - Meta (medium)
  • Should WhatsApp launch group calls? - Meta (medium)
  • How would you grow Meta products? - Meta (medium)
Meta logo
Meta
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
3
0

Production Incident: 10% Drop in Daily Likes (DAU Flat) on 2025-09-01

You are investigating a 10% day-over-day drop in daily Like actions on a global social network on 2025-09-01, while DAU remained flat versus the prior 14-day average. Treat this as a production incident.

Provide an end-to-end diagnostic plan that includes:

  1. Data validation checks
    • Enumerate concrete checks (e.g., event volume parity vs app_sessions, null-rate and schema-change diffs, client→server receipt ratios, deduplication rates, timestamp skew, ingestion delay, endpoint error rates).
    • Define specific thresholds that strongly indicate instrumentation failure vs normal noise.
  2. Expected variability and exogenous factors
    • How you will rule out seasonality and expected variability (day-of-week, month-start effects), regional holidays, release calendars, and major outages/events.
    • Specify what external data you would join and how.
  3. Localization
    • Localize the drop by funnel stage: feed impressions → post views → like impressions → like actions. Compute stage-to-stage conversion rates.
    • Localize by segments: surface (friends vs pages vs events), platform (iOS/Android/Web), app version, geography, new vs retained users, time-of-day.
  4. Hypotheses and tests
    • State three falsifiable hypotheses for the drop.
    • For each, list the exact cuts/queries/plots you will run to test them.
  5. Decision tree to root cause (≤24 hours)
    • A concrete decision tree that moves from validation → external factors → localization → root cause.
    • Include what you’d do if multiple segments move in opposite directions (mix-shift/Simpson’s paradox).
  6. Mitigation and prevention
    • One immediate mitigation (safe to deploy quickly).
    • One follow-up experiment/process to prevent recurrence.

Finally, list the first three queries or charts you would run and the precise outcomes that would make you conclude that behavior change (not logging) is at fault.

Solution

Show

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.