Choose Randomization Unit and Mitigate Network Effects

Q: Choose Randomization Unit and Mitigate Network Effects

This question evaluates a candidate's competency in experimental design and causal inference, specifically selecting appropriate randomization units, detecting and mitigating network interference, and estimating direct versus spillover effects.

Q: How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

Question

A/B Test Design for a New Messenger Feature with Network Effects

Context

You are designing an A/B test for a new Messenger feature in a social messaging app. Users communicate via one-to-one and group conversations. Because people interact, there is a risk that a participant's assignment could influence their peers' outcomes (network interference), violating the standard "no interference" assumption of A/B tests.

Tasks

Choose an appropriate randomization unit (e.g., user-level, conversation/thread-level, cluster/graph-level) and justify your choice given potential network effects.
Describe practical diagnostics and statistical tests you would use to detect network effects if they occur.
Propose design and analysis methods to mitigate bias from network effects and to estimate both direct and spillover effects. Where useful, mention methods such as CUPED and difference-in-differences.

Choose Randomization Unit and Mitigate Network Effects

A/B Test Design for a New Messenger Feature with Network Effects

Context

Tasks

Solution

Comments (0)