The metric "average number of posts per DAU" drops from 3.0 to 2.5 between 2025-08-31 and 2025-09-01. 1) List and prioritize at least 8 concrete hypotheses spanning product, data, seasonality, and traffic-mix causes (e.g., UI change in the composer flow, posting frictions, rate limits, instrumentation errors, bot mitigation, geo mix shift, app version rollout, outage). For each, specify one falsifiable check and the exact slice you would inspect. 2) Design a same-day triage plan to distinguish measurement bugs from true behavior change: what counters, logs, or A/A checks would you run, and what thresholds decide a bug? 3) If a new feature shipped on 2025-09-01 in the composer, propose an A/B test or phased rollout plan to judge if the change is good or bad: define primary metric(s), at least three guardrail metrics (e.g., crash rate, DAU retention D+1, session length), and key segments (country, platform, new vs returning). 4) Specify decision criteria (minimum effect size and statistical thresholds) and the data collection window you would choose to avoid day-of-week bias. 5) Describe how you would validate that country-level impacts are not masking a global average (Simpson’s paradox) and what action you would take if one large country drives the drop.

This question evaluates a data scientist's competency in diagnostic analytics, instrumentation validation, causal attribution, experimentation design, and statistical reasoning (including detection of Simpson’s paradox) for product metrics.

How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Diagnose drop and assess metric change impact

Investigate a Drop in Average Posts per DAU

Context

You work on a large consumer social app. The metric "average number of posts per DAU" (daily active users) is defined as:

posts_per_DAU = total_posts_in_day / DAU_in_day

Between 2025-08-31 and 2025-09-01, this metric fell from 3.0 to 2.5 (−16.7%). Assume the definition excludes obvious spam/bot posts and covers all first-party posting surfaces captured by your post-create pipeline.

Tasks

List and prioritize at least 8 concrete hypotheses spanning product, data/instrumentation, seasonality, and traffic-mix causes. For each hypothesis, provide:
- One falsifiable check you would run immediately.
- The exact slice(s) you would inspect to validate or reject it.
Design a same-day triage plan to distinguish measurement bugs from true behavior change. Specify:
- The counters, logs, or A/A checks you would run.
- Quantitative thresholds that would lead you to conclude a measurement bug.
If a new composer feature shipped on 2025-09-01, propose an A/B test or phased rollout plan to judge whether the change is good or bad. Define:
- Primary metric(s) and at least three guardrail metrics.
- Key segments (e.g., country, platform, new vs. returning users).
Specify decision criteria (minimum effect size and statistical thresholds) and a data collection window that avoids day-of-week bias.
Describe how you would validate that country-level impacts are not masking a global average (Simpson’s paradox). State what action you would take if one large country drives the drop.

Context

You work on a large consumer social app. The metric "average number of posts per DAU" (daily active users) is defined as:

posts_per_DAU = total_posts_in_day / DAU_in_day

Tasks

List and prioritize at least 8 concrete hypotheses spanning product, data/instrumentation, seasonality, and traffic-mix causes. For each hypothesis, provide:

One falsifiable check you would run immediately.
The exact slice(s) you would inspect to validate or reject it.

Design a same-day triage plan to distinguish measurement bugs from true behavior change. Specify:

The counters, logs, or A/A checks you would run.
Quantitative thresholds that would lead you to conclude a measurement bug.

If a new composer feature shipped on 2025-09-01, propose an A/B test or phased rollout plan to judge whether the change is good or bad. Define:

Primary metric(s) and at least three guardrail metrics.
Key segments (e.g., country, platform, new vs. returning users).

Specify decision criteria (minimum effect size and statistical thresholds) and a data collection window that avoids day-of-week bias.

Describe how you would validate that country-level impacts are not masking a global average (Simpson’s paradox). State what action you would take if one large country drives the drop.

Diagnose drop and assess metric change impact

Quick Overview

Investigate a Drop in Average Posts per DAU

Context

Tasks

Solution

Submit Your Answer to Earn 20XP

Diagnose drop and assess metric change impact

Quick Overview

Investigate a Drop in Average Posts per DAU

Context

Tasks

Solution

Submit Your Answer to Earn 20XP