How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Evaluate Success of 'Similar Listings' Notification Feature

Quick Overview

This question evaluates a data scientist's competency in product analytics, causal inference, experimental design, and metric-definition for a notification feature that surfaces listings similar to items a user has viewed or saved, situated in the Analytics & Experimentation domain.

Marketplace Analytics Case: "Similar Listings You May Like" Notifications

Context

You work on a marketplace where buyers browse and purchase listings. The team is considering a new notification feature (push/email/in-app) that alerts users when new listings similar to items they've viewed or saved become available. The goal is to increase user engagement and conversion without causing notification fatigue or harming overall marketplace health.

Tasks

Decision framework: How would you decide whether introducing this notification feature is a good idea? State your hypotheses, expected user/business value, risks, and prerequisites.
Experiment design: Propose an A/B test to evaluate the feature. Specify unit of randomization, eligibility, randomization scheme, sample size/power, duration, attribution windows, and a roll-out plan (ramp/holdouts).
Success criteria: Define and justify primary, secondary, and guardrail metrics (e.g., CTR, conversion, retention, user satisfaction), including precise metric definitions and evaluation windows.

Quick Overview

Context

Tasks

Decision framework: How would you decide whether introducing this notification feature is a good idea? State your hypotheses, expected user/business value, risks, and prerequisites.

Experiment design: Propose an A/B test to evaluate the feature. Specify unit of randomization, eligibility, randomization scheme, sample size/power, duration, attribution windows, and a roll-out plan (ramp/holdouts).

Success criteria: Define and justify primary, secondary, and guardrail metrics (e.g., CTR, conversion, retention, user satisfaction), including precise metric definitions and evaluation windows.

Evaluate Success of 'Similar Listings' Notification Feature

Quick Overview

Marketplace Analytics Case: "Similar Listings You May Like" Notifications

Context

Tasks

Solution

Submit Your Answer to Earn 20XP

Evaluate Success of 'Similar Listings' Notification Feature

Quick Overview

Marketplace Analytics Case: "Similar Listings You May Like" Notifications

Context

Tasks

Solution

Submit Your Answer to Earn 20XP