Diagnose a sudden KPI drop
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: Medium
Interview Round: Onsite
On 2025-09-01, a global social network observes a 10% decline in daily Likes per DAU versus the prior 4-week same-weekday baseline. Walk me through a rigorous diagnosis plan:
- What immediate data-quality and instrumentation checks would you run (e.g., event volume parity, client/server log parity, duplication, anti-spam filters, pipeline lags)?
- How would you de-seasonalize and choose the right counterfactual (same weekday, time-of-day mix, holiday/outage exclusions)?
- Which segmentations would you prioritize (country, platform, app version, post_type, new vs. tenured users, acquisition channel), and why? Specify the exact metrics and plots you’d produce.
- How would you rule in/out external events and internal changes (releases, experiments, config flags)?
- Propose two fast hypotheses that could explain a Likes drop without a DAU drop, and design minimal tests to validate them within the same day.
- Define guardrail metrics and stop-loss thresholds for any mitigations you’d roll out while investigating.
Deliver a prioritized action plan you could execute in 2 hours, including the first three queries/analyses you would run and what decisions each would unblock.
Quick Answer: This question evaluates operational analytics and experimentation competencies, including instrumentation and data-quality checks, de-seasonalization and counterfactual selection, segmentation-based root-cause analysis, causal attribution, rapid hypothesis generation, and mitigation guardrails within the Analytics & Experimentation domain.