How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Select and prioritize metrics with guardrails

Quick Overview

This question evaluates a candidate's ability to design a robust metrics framework for product experiments, covering selection and normalization of success metrics, feature-only sizing and ecosystem metrics, explicit cannibalization reads, guardrails, multiple-testing control, and back-of-the-envelope LTV/ARPDAU impact estimation.

Design a Metrics Framework for a New Groups Stories Feature

Context

You are evaluating a new Groups Stories feature whose goal is to increase meaningful engagement while avoiding harm to the broader product ecosystem. Propose a metrics framework for experimentation and rollout gating, including success metrics, guardrails, and ecosystem checks.

Tasks

Define three top-line goal (success) metrics and specify the normalization base (per DAU, per viewer, or per poster) and why.
List feature-only sizing metrics (e.g., treatment-only CTR, creation rate) to estimate volumes if launched.
List ecosystem metrics to catch second-order effects (e.g., sessions, retention, revenue).
Specify explicit cannibalization reads on other surfaces (e.g., Feed, Marketplace, Pages).
Define guardrails (e.g., latency, crash, report rate, cancellations) and sanity checks.
Specify a multiple-testing control method (e.g., Holm–Bonferroni vs Benjamini–Hochberg) and when you’d prefer each.
Show a back-of-the-envelope calculation for LTV and ARPDAU impact from a +1% DAU movement.

Finally, pick a minimal gating set (three metrics that must be green to roll out to 100%) and explain why the other metrics are secondary.

Quick Overview

Tasks

Define three top-line goal (success) metrics and specify the normalization base (per DAU, per viewer, or per poster) and why.

List feature-only sizing metrics (e.g., treatment-only CTR, creation rate) to estimate volumes if launched.

List ecosystem metrics to catch second-order effects (e.g., sessions, retention, revenue).

Specify explicit cannibalization reads on other surfaces (e.g., Feed, Marketplace, Pages).

Define guardrails (e.g., latency, crash, report rate, cancellations) and sanity checks.

Specify a multiple-testing control method (e.g., Holm–Bonferroni vs Benjamini–Hochberg) and when you’d prefer each.

Show a back-of-the-envelope calculation for LTV and ARPDAU impact from a +1% DAU movement.

Finally, pick a minimal gating set (three metrics that must be green to roll out to 100%) and explain why the other metrics are secondary.

Select and prioritize metrics with guardrails

Quick Overview

Design a Metrics Framework for a New Groups Stories Feature

Context

Tasks

Solution

Submit Your Answer to Earn 20XP

Select and prioritize metrics with guardrails

Quick Overview

Design a Metrics Framework for a New Groups Stories Feature

Context

Tasks

Solution

Submit Your Answer to Earn 20XP