How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Analyze Key Metrics for Notification System Success

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's ability to select and interpret key product metrics, reason about trade-offs between short-term engagement and medium-term retention, and analyze randomized A/B experiment outcomes to detect user annoyance or churn.

Scenario

You are evaluating a new push-notification system for a social app. The goal is to determine whether the new system improves user value without increasing user annoyance or churn.

Assume you can run a randomized A/B experiment (50/50 at the user level), collect notification- and user-level events, and observe metrics for at least 2–4 weeks. Consider both short-term engagement and medium-term retention outcomes.

Questions

Define the key metrics you would track to measure notification quality.
How would you decide whether to launch the new notification experience?
If one metric improves while another declines after the experiment, how would you analyze and reconcile the discrepancy?

Hints: Consider engagement, retention, churn, user annoyance; design an A/B test and deep-dive segment analysis.

Solution

Show

Submit Your Answer to Earn 20XP

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation