Evaluates product metric diagnosis for a sudden 10% DAU decline in Yahoo Mail. Strong answers validate measurement, inspect mail funnels and cohorts, segment by platform and geography, rule out releases or incidents, and separate internal regressions from external factors.
##### Scenario
Yahoo Mail suddenly reports a 10 % decline in daily active users (DAU).
##### Question
Walk me through how you would investigate and diagnose the root cause of the 10 % DAU drop for Yahoo Mail. What data would you pull, which funnels or cohorts would you examine, and how would you rule out internal versus external factors?
##### Hints
Form a hypothesis tree, segment traffic by platform, geography, user cohorts; inspect release logs, outages, marketing, seasonality, competitor news.
Quick Answer: Evaluates product metric diagnosis for a sudden 10% DAU decline in Yahoo Mail. Strong answers validate measurement, inspect mail funnels and cohorts, segment by platform and geography, rule out releases or incidents, and separate internal regressions from external factors.
Yahoo Mail suddenly reports a 10% decline in daily active users. You need to investigate whether the drop is real, localize it, and identify likely causes.
Constraints & Assumptions
Treat this as a live product metric diagnosis.
Verify measurement integrity before assuming user behavior changed.
Assume DAU is based on unique accounts with at least one qualifying session or mail event.
Consider internal causes such as releases, incidents, logging, experiments, and external causes such as seasonality or ecosystem changes.
Clarifying Questions to Ask Guidance
When did the 10% decline start, and did it happen suddenly or gradually?
Is the decline in all DAU or only one platform, market, user type, or event definition?
Were there recent releases, experiments, outages, marketing changes, or tracking changes?
Is the drop visible in server-side logs as well as client analytics?
Part 1 - Pull Data and Validate the Metric
What data would you pull first?
What This Part Should Cover Guidance
Client and server DAU, login, inbox view, email read, compose/send, notification, and session logs.
Data latency, backfills, event schema changes, bot filters, timezone boundaries, and duplicate account handling.
Incident, release, experiment, feature-flag, marketing, support, and external ecosystem data.
Part 2 - Examine Funnels and Cohorts
Which funnels, cohorts, and segments would you examine to localize the issue?
Web, iOS, Android, app version, browser, geography, ISP, new versus returning users, tenure, and account type.
Contribution analysis to find which segment explains the aggregate decline.
Part 3 - Rule Out Causes
How would you rule out internal versus external factors?
What This Part Should Cover Guidance
Correlate timing with releases, outages, experiments, partner changes, or logging changes.
Compare holdouts, unaffected markets, previous weeks, same weekday, and external benchmarks.
Use rollback, difference-in-differences, synthetic control, or incident analysis where feasible.
What a Strong Answer Covers Guidance
A strong answer uses a hypothesis tree, validates the metric, segments the decline, separates product or infrastructure regressions from external effects, and recommends immediate mitigation plus follow-up validation.
Follow-up Questions Guidance
What if the decline appears only on iOS after an app release?
How would you handle a logging bug that affects DAU but not real usage?
What would you put in the executive incident update?