Determine Value of Prioritizing Accounts by Unread Notifications

Quick Overview

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Determine Value of Prioritizing Accounts by Unread Notifications states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Determine Value of Prioritizing Accounts by Unread Notifications

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

##### Scenario A product team wants to introduce a feature that, for users with multiple accounts, orders those accounts by descending count of unread notifications. ##### Question How would you determine, using data, whether prioritizing accounts by unread-notification count is a worthwhile idea? Design an A/B experiment to validate this feature. Specify population, treatment, control, success metrics, and test duration. Imagine the test shows that the frequency of account switching drops while the total number of active accounts rises. How would you interpret this result and decide next steps? ##### Hints Think about exploratory historical analysis, customer behavior segmentation, primary vs. guard-rail metrics, experiment power, and reconciling conflicting short-term and long-term signals.

Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Determine Value of Prioritizing Accounts by Unread Notifications states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Aug 4, 2025, 10:55 AM
mediumData ScientistTechnical ScreenAnalytics & Experimentation
5
0

Determine Value of Prioritizing Accounts by Unread Notifications

Feature Validation: Ordering Multiple Accounts by Unread Notifications

Context

Users can hold multiple accounts and switch between them. The product team proposes to order a user's accounts by descending count of unread notifications to help them find the next most relevant account quickly.

Definitions (for clarity):

  • Account switching: The user changes from one account to another within a session.
  • Active account (per period): An account with at least one meaningful action (e.g., session, post, message, read/clear notification) in the period.

Tasks

  1. Using data, outline how you would assess whether prioritizing accounts by unread-notification count is a worthwhile idea before shipping (e.g., exploratory/historical analyses, segmentation).
  2. Design an A/B test to validate the feature. Specify:
    • Population and unit of randomization
    • Treatment and control
    • Primary and guardrail metrics (with rationale)
    • Test duration and power considerations
  3. The hypothetical result shows: frequency of account switching drops, while total number of active accounts rises. Interpret this outcome and propose next steps.

Hints

Consider exploratory historical analysis, customer behavior segmentation, primary vs. guard-rail metrics, experiment power, and how to reconcile conflicting short-term and long-term signals.

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
  • State assumptions about instrumentation, randomization, sample size, and data quality.
  • Separate descriptive analysis from causal claims.

What a Strong Answer Covers Guidance

  • A metric framework with primary, guardrail, and diagnostic metrics.
  • A credible analysis or experiment design with clear assumptions and bias checks.
  • SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
  • An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions Guidance

  • What sanity checks would you run before trusting the result?
  • How would you handle novelty effects, seasonality, or selection bias?
  • What decision would you make if metrics disagree?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation