Implement Clustered Sampling to Mitigate Network Effects in Testing
Overview
This question evaluates a candidate's ability to design clustered randomization strategies to address interference and spillovers in networked experiments, testing competencies in causal inference, experimental design, and statistical reasoning within Analytics & Experimentation.
Meta
Jul 12, 2025, 6:59 PM
Data Scientist
Technical Screen
Analytics & Experimentation
15
0
Scenario
You are planning an A/B test for a new recommendation algorithm in a networked product where users interact with friends and creators. Because user outcomes can be affected by others' assignments (network spillovers), standard individual randomization may violate the "no-interference" assumption.
Task
Design a clustered random sampling approach to select test and control users that limits interference.
Questions
- How would you construct clusters and assign treatment/control at the cluster level?
- Which clustering choices make sense in this context (e.g., markets, friend/interaction graphs), and why do they reduce interference?
- What trade-offs or risks does clustered sampling introduce (e.g., intracluster correlation and power loss)?
Hints
- Consider geographic/language markets and social/interaction graphs.
- Aim to minimize cross-cluster ties that carry spillovers.
- Intracluster correlation (ICC) inflates variance; anticipate power loss.
- Think about analysis with cluster-robust methods and stratification.
Solution
Login required
Show