How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Design a small-sample launch experiment in Europe

Quick Overview

This question evaluates a data scientist's competencies in experimental design, causal inference, estimand specification, power and MDE calculations for heavy-tailed and overdispersed outcomes, interference handling, compliance/IV considerations, and sequential monitoring in field experiments.

Launch Test Design: Early-Access EU Businesses

Context

You have an early-access pool of 1,200 EU businesses for a new chat subscription offering. Chat volumes are heavy-tailed. You expect 20–30% initial subscription take-up among those offered early access.

You need to estimate the causal impact of the launch on:

Subscription revenue per business.
Resolved-Within-24h Rate (share of chats resolved within 24 hours).

There are interference risks because some customers may interact with multiple businesses. You must choose and justify one design among: a cluster randomized controlled trial (business-level or region-level), a stepped-wedge rollout, or a quasi-experiment.

Task

Design the launch test and provide a full analysis plan, including:

Choice and justification of design (cluster RCT at business-level or region-level, stepped-wedge, or quasi-experiment).
Unit of randomization and stratification variables (country, size, baseline volume).
Handling heavy-tailed outcomes and overdispersion in power/MDE calculations.
Interference risks due to shared customers across businesses.
Compliance and adoption treatment (ITT vs TOT) and IV approach.
Sequential monitoring and alpha spending.
A pre-registered primary analysis plan: estimands, modeling choices, covariate adjustment, heterogeneity (region/vertical), missing data rules.
A decision rule to roll out, iterate, or stop.

Quick Overview

Context

You need to estimate the causal impact of the launch on:

Subscription revenue per business.

Resolved-Within-24h Rate (share of chats resolved within 24 hours).

Task

Design the launch test and provide a full analysis plan, including:

Choice and justification of design (cluster RCT at business-level or region-level, stepped-wedge, or quasi-experiment).

Unit of randomization and stratification variables (country, size, baseline volume).

Handling heavy-tailed outcomes and overdispersion in power/MDE calculations.

Interference risks due to shared customers across businesses.

Compliance and adoption treatment (ITT vs TOT) and IV approach.

Sequential monitoring and alpha spending.

A pre-registered primary analysis plan: estimands, modeling choices, covariate adjustment, heterogeneity (region/vertical), missing data rules.

A decision rule to roll out, iterate, or stop.

Design a small-sample launch experiment in Europe

Quick Overview

Launch Test Design: Early-Access EU Businesses

Context

Task

Solution

Submit Your Answer to Earn 20XP

Design a small-sample launch experiment in Europe

Quick Overview

Launch Test Design: Early-Access EU Businesses

Context

Task

Solution

Submit Your Answer to Earn 20XP