How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Design experiments under network interference

Quick Overview

This question evaluates a data scientist's competence in experimental design and causal inference under network interference, assessing understanding of clustering strategies, exposure and exclusion definitions, contamination control, design effect and sample-size implications, estimation of intra-cluster correlation, and mixed-effects analysis for evolving clusters. Commonly asked in Analytics & Experimentation interviews to probe the ability to preserve internal validity while managing operational feasibility in two-sided marketplaces, it tests both conceptual understanding of interference and practical application skills in design and analysis planning.

A/B Test Design for Search-Ranking in a Two-Sided Marketplace with Interference

Context

You need to evaluate a change to the search-ranking algorithm in a two-sided marketplace where:

Sellers can affect each other (cross-seller spillovers, e.g., inventory or ranking shifts that alter other sellers' exposure).
Consumers can influence each other (social contagion, e.g., shares, word-of-mouth).

Design an interference-aware experiment that preserves internal validity while remaining operationally feasible.

Tasks

Choose a randomization strategy: cluster randomization (e.g., by seller or geography) vs. ego-cluster randomization. Justify the choice given cross-seller spillovers and consumer social contagion.
Precisely define exposure and exclusion criteria to control interference.
Compute design effect and sample-size inflation for clustering with mean cluster size m = 50 and intra-cluster correlation ρ = 0.02.
Explain how to estimate ρ pre-experiment from historical data.
Describe how to prevent treatment contamination during reassignment and how to validate success using invariant metrics and sample-size ratio mismatch (SSRM) under clustering.
If clusters evolve over time, propose a rehashing (re-clustering) schedule and an analysis plan using mixed-effects models with cluster random intercepts.

Quick Overview

Context

You need to evaluate a change to the search-ranking algorithm in a two-sided marketplace where:

Sellers can affect each other (cross-seller spillovers, e.g., inventory or ranking shifts that alter other sellers' exposure).

Consumers can influence each other (social contagion, e.g., shares, word-of-mouth).

Design an interference-aware experiment that preserves internal validity while remaining operationally feasible.

Tasks

Choose a randomization strategy: cluster randomization (e.g., by seller or geography) vs. ego-cluster randomization. Justify the choice given cross-seller spillovers and consumer social contagion.

Precisely define exposure and exclusion criteria to control interference.

Compute design effect and sample-size inflation for clustering with mean cluster size m = 50 and intra-cluster correlation ρ = 0.02.

Explain how to estimate ρ pre-experiment from historical data.

Describe how to prevent treatment contamination during reassignment and how to validate success using invariant metrics and sample-size ratio mismatch (SSRM) under clustering.

If clusters evolve over time, propose a rehashing (re-clustering) schedule and an analysis plan using mixed-effects models with cluster random intercepts.

Design experiments under network interference

Quick Overview

A/B Test Design for Search-Ranking in a Two-Sided Marketplace with Interference

Context

Tasks

Solution

Submit Your Answer to Earn 20XP

Design experiments under network interference

Quick Overview

A/B Test Design for Search-Ranking in a Two-Sided Marketplace with Interference

Context

Tasks

Solution

Submit Your Answer to Earn 20XP