Design experiments under network interference

Q: Design experiments under network interference

This question evaluates a data scientist's competence in experimental design and causal inference under network interference, assessing understanding of clustering strategies, exposure and exclusion definitions, contamination control, design effect and sample-size implications, estimation of intra-cluster correlation, and mixed-effects analysis for evolving clusters. Commonly asked in Analytics & Experimentation interviews to probe the ability to preserve internal validity while managing operational feasibility in two-sided marketplaces, it tests both conceptual understanding of interference and practical application skills in design and analysis planning.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Question

A/B Test Design for Search-Ranking in a Two-Sided Marketplace with Interference

Context

You need to evaluate a change to the search-ranking algorithm in a two-sided marketplace where:

Sellers can affect each other (cross-seller spillovers, e.g., inventory or ranking shifts that alter other sellers' exposure).
Consumers can influence each other (social contagion, e.g., shares, word-of-mouth).

Design an interference-aware experiment that preserves internal validity while remaining operationally feasible.

Tasks

Choose a randomization strategy: cluster randomization (e.g., by seller or geography) vs. ego-cluster randomization. Justify the choice given cross-seller spillovers and consumer social contagion.
Precisely define exposure and exclusion criteria to control interference.
Compute design effect and sample-size inflation for clustering with mean cluster size m = 50 and intra-cluster correlation ρ = 0.02.
Explain how to estimate ρ pre-experiment from historical data.
Describe how to prevent treatment contamination during reassignment and how to validate success using invariant metrics and sample-size ratio mismatch (SSRM) under clustering.
If clusters evolve over time, propose a rehashing (re-clustering) schedule and an analysis plan using mixed-effects models with cluster random intercepts.

Design experiments under network interference

Quick Overview

Design experiments under network interference

A/B Test Design for Search-Ranking in a Two-Sided Marketplace with Interference

Context

Tasks

Write your answer