PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Meta

Design experiment for Group Calls with interference

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's competency in experimental design and causal inference for networked features, covering metrics selection (primary and guardrail metrics), strategies for mitigating interference via randomization or quasi-experimental approaches, exposure and contamination controls, rollout planning, and pre-registration and power analysis. Commonly asked in Analytics & Experimentation interviews for Data Scientist roles, it assesses reasoning about bias–variance trade-offs, engineering constraints, and validity diagnostics in interconnected user environments, requiring both conceptual understanding of identification and practical application of experiment implementation.

  • hard
  • Meta
  • Analytics & Experimentation
  • Data Scientist

Design experiment for Group Calls with interference

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Your team is adding a Group Calls feature to a 1:1 calling app. Design a robust experiment to measure its impact under likely network interference. Address: (a) Select exactly one primary metric and 3–5 guardrails; explain when "total call volume" is misleading (e.g., cannibalization, unequal exposure) and when it could be acceptable; propose user-normalized or rate-based alternatives. (b) Choose a randomization unit (e.g., graph clusters, ego-clusters, geo switchbacks) and a rollout plan that mitigates interference; compare bias, variance, and engineering complexity trade-offs. (c) If cluster-based randomization is not feasible, propose a quasi-experimental plan: e.g., invitation-gated treatment, time-based/switchback assignment, exposure-weighted estimators, IV using staggered access, or difference-in-differences with pre-periods; list assumptions and diagnostics you will run (placebo tests, balance checks, spillover detection). (d) Define exposure, triggers, and exclusion criteria (e.g., creators vs joiners; first exposure vs ever-exposed); detail contamination controls (e.g., invite tokens only visible to treated egos). (e) Pre-register analysis, specify minimal detectable effect with plausible baselines, seasonality controls, and how you'll handle novelty and ramp-up effects.

Quick Answer: This question evaluates a data scientist's competency in experimental design and causal inference for networked features, covering metrics selection (primary and guardrail metrics), strategies for mitigating interference via randomization or quasi-experimental approaches, exposure and contamination controls, rollout planning, and pre-registration and power analysis. Commonly asked in Analytics & Experimentation interviews for Data Scientist roles, it assesses reasoning about bias–variance trade-offs, engineering constraints, and validity diagnostics in interconnected user environments, requiring both conceptual understanding of identification and practical application of experiment implementation.

Related Interview Questions

  • Measure scheduled posts feature success - Meta (medium)
  • Estimate ads ranking revenue impact - Meta (medium)
  • How should you evaluate unconnected content? - Meta (medium)
  • Should WhatsApp launch group calls? - Meta (medium)
  • How would you grow Meta products? - Meta (medium)
Meta logo
Meta
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0

Design an Experiment for Group Calls in a 1:1 Calling App (with Network Interference)

You are adding a Group Calls feature to an existing 1:1 calling app. Design a robust experiment to measure the feature's impact when users can influence each other (network interference is likely).

Assume a large, global user base and that group calls allow a user to create a group and invite multiple contacts. Some users might only join via invites. You may gate creation or visibility to mitigate contamination.

Address the following:

(a) Metrics

  • Select exactly one primary metric and 3–5 guardrail metrics.
  • Explain when "total call volume" is misleading (e.g., cannibalization, unequal exposure) and when it could be acceptable.
  • Propose user-normalized or rate-based alternatives to total call volume.

(b) Randomization Unit and Rollout

  • Choose a randomization unit to mitigate interference (e.g., graph clusters, ego-clusters, geo switchbacks).
  • Propose a rollout plan.
  • Compare bias, variance, and engineering complexity trade-offs across options.

(c) If Cluster Randomization Is Not Feasible

  • Propose a quasi-experimental plan: e.g., invitation-gated treatment, time-based/switchback assignment, exposure-weighted estimators, instrumental variables using staggered access, or difference-in-differences with pre-periods.
  • List identifying assumptions and diagnostics (placebo tests, balance checks, spillover detection).

(d) Exposure, Triggers, and Contamination Controls

  • Define exposure and triggers (creators vs. joiners; first exposure vs. ever-exposed).
  • Specify exclusion criteria.
  • Detail contamination controls (e.g., invite tokens only visible to treated egos).

(e) Pre-registration and Power

  • Pre-register the analysis: hypotheses, windows, estimators, and stopping rules.
  • Specify a minimal detectable effect (MDE) with plausible baselines.
  • Include seasonality controls and how you'll handle novelty/ramp-up effects.

Solution

Show

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.