How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Evaluate Instagram's Short-Video Recommender System Success

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's ability to choose and justify a primary product metric, reason about distributional properties and guardrail metrics, and design end-to-end A/B tests for a short-video recommender feed.

Evaluating a New Short‑Video Recommender Feed

Context

You are a data scientist preparing metrics and an A/B test plan to launch a new short‑video recommender feed within Instagram. The goal is to measure whether the ranking algorithm improves user value and is safe to ship.

Questions

(a) Which single metric would you use to evaluate the recommendation system’s success, and why?

(b) Sketch or describe the expected distribution of that metric, labeling the median, mode, and 95th percentile.

(d) List the end‑to‑end A/B‑testing steps for this launch.

Hints

Discuss engagement vs retention
Heavy‑tailed distributions
Guardrail metrics
Power calculation and success criteria

Solution

Show

Submit Your Answer

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation