Design A/B Test for Streaming Feature Network Effects

Q: Design A/B Test for Streaming Feature Network Effects

This question evaluates a candidate's understanding of online experiment design and causal inference under network spillovers, covering metric selection, experimental unit definition, SUTVA, variance-reduction techniques, multivariate testing, and heterogeneous treatment effects.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Context

You are designing and evaluating an online experiment for a new social/streaming feature (e.g., sharing, co-watching, reactions) that can influence users' friends, creating spillovers across the social graph.

Assume:

The platform has user accounts, some nested in households/profiles.
Users can affect each other through in-app social features (network effects).
You will also run a multivariate test (MVT) on title artwork to improve content discovery.

Questions

How would you choose primary and guardrail metrics for this experiment?
What threats do network effects pose and how would you address them?
Explain SUTVA and why it matters here.
How would you run and interpret a multivariate test on title artwork?
How would you measure heterogeneous treatment effects (HTE) across user segments?

HINTS: Cover metric definition, experiment unit, variance reduction, and validity assumptions.

Design A/B Test for Streaming Feature Network Effects

Context

Questions

Solution

Comments (0)

Design A/B Test for Streaming Feature Network Effects

Overview

A/B Test Design With Potential Social-Network Spillovers (Streaming Platform)

Context

Questions

Solution

Comments (0)