PracHub
QuestionsPremiumLearningGuidesInterview PrepCoaches
|Home/Analytics & Experimentation/Roblox

Evaluate friend-interaction feature with network interference

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's competency in causal inference and network-aware experiment design, covering randomization under interference, exposure-weighted metric specification, power estimation with intracluster correlation, and cluster-robust analysis.

  • hard
  • Roblox
  • Analytics & Experimentation
  • Data Scientist

Evaluate friend-interaction feature with network interference

Company: Roblox

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: HR Screen

You plan to ship a “friend interaction boost” that ranks feed content higher when friends interact with it. Because interactions propagate through the social graph, standard user-level randomization violates SUTVA. Design an experiment that credibly estimates causal lift: 1) Randomization unit: Choose and defend graph-cluster randomization (e.g., Louvain clusters) vs ego-network clustering vs geo/time switchbacks. How will you quantify and cap cross-cluster edge cut ratio and exposure contamination? 2) Metrics: Define primary (session time, meaningful interactions) and guardrail metrics (spam reports, creator revenue cannibalization). Specify exposure-weighted metrics for partially treated users. 3) Power: Given average cluster size m and intracluster correlation ρ, derive effective sample size n_eff ≈ (K·m) / [1+(m−1)ρ]. Show how this changes required duration vs user-level AB. 4) Analysis: Outline cluster-robust variance estimation, CUPED with pre-period outcomes, and intent-to-treat vs exposure-on-treated estimands. How do you handle creators whose audiences span treatment/control clusters? 5) Diagnostics & fallbacks: Pre-commit spillover checks, negative controls, and a holdout of high-degree nodes. If contamination is too high mid-test, propose a redesign that preserves inference while limiting blast radius.

Quick Answer: This question evaluates a data scientist's competency in causal inference and network-aware experiment design, covering randomization under interference, exposure-weighted metric specification, power estimation with intracluster correlation, and cluster-robust analysis.

Related Interview Questions

  • How to estimate feature impact on usage time - Roblox (easy)
  • How to estimate a feature’s causal impact on time spent - Roblox (medium)
  • Compute DID estimate and pretrend flag - Roblox (hard)
  • Compute minimum sample size for A/B test - Roblox (hard)
  • Compute DiD and validate parallel trends - Roblox (hard)
Roblox logo
Roblox
Oct 13, 2025, 9:49 PM
Data Scientist
HR Screen
Analytics & Experimentation
3
0

Experiment Design: Network-Aware Test for a "Friend Interaction Boost" in Feed Ranking

Context

You plan to ship a ranking change that boosts items in the feed when a user's friends have interacted with them. Because interactions propagate through the social graph, the Stable Unit Treatment Value Assumption (SUTVA) is violated under user-level randomization: a user's outcome can be affected by neighbors' assignments. Design a credible experiment that estimates causal lift under network interference.

Tasks

  1. Randomization Unit
  • Choose and defend one of: graph-cluster randomization (e.g., Louvain or partitioning), ego-network clustering, or geo/time switchbacks.
  • Define how you will quantify and cap:
    • Cross-cluster edge cut ratio.
    • Exposure contamination across treatment arms.
  1. Metrics
  • Define primary metrics (e.g., session time, meaningful interactions) and guardrail metrics (e.g., spam reports, creator revenue cannibalization).
  • Specify exposure-weighted metrics for partially treated users.
  1. Power
  • Given average cluster size m and intracluster correlation ρ, derive the effective sample size: n_eff ≈ (K · m) / [1 + (m − 1)ρ] where K is the number of clusters.
  • Explain how this changes required test duration relative to a user-level A/B test.
  1. Analysis
  • Outline cluster-robust variance estimation; use CUPED with pre-period outcomes.
  • Define intent-to-treat (ITT) vs exposure-on-treated estimands and how to estimate each.
  • Describe how to handle creators whose audiences span both treatment and control clusters.
  1. Diagnostics and Fallbacks
  • Pre-commit spillover checks, negative controls, and a holdout of high-degree nodes.
  • If contamination is too high mid-test, propose a redesign that preserves inference while limiting blast radius.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Roblox•More Data Scientist•Roblox Data Scientist•Roblox Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.