Design analytics and experiment for group video calls
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
You are evaluating a new Group Video Call feature in a messaging app. Using only product analytics (no user surveys), answer the following in depth:
1) Analyze relationships between users: Given access to historical messaging and 1:1 call data, propose a concrete plan to quantify tie strength and group affinity to identify who actually needs group calls. Specify at least five metrics (e.g., mutual message frequency, recency-weighted interaction score, common-neighbor count/triadic closure, thread-size distribution, co-call co-occurrence, clustering coefficient). Describe how you would segment users/threads and produce a ranked list of candidate groups to target at launch.
2) Instrumentation wishlist (recipient-focused): You cannot run a survey. Draft the exact event logs you want engineering to add to infer the recipient’s need for a group call. For each event (e.g., recipient sees incoming group call, attempts to add >1 participant from a 1:1 thread, attempts multi-dial, abandons add-participant flow, retries after failure), specify: event name, triggering UI action, required fields (user_id, thread_id, call_id, selected_participant_count, addressable_friend_count, prior_missed_calls_7d, network_type, device_os, error_code, latency_ms, local_time), and sampling/PII constraints. Explain how you would transform these logs into leading-indicator metrics of latent demand.
3) Choose a Meta proxy product: Pick one existing Meta product (e.g., Messenger, WhatsApp, Instagram, or Workplace) to use as a proxy for forecasting adoption and operational risks. Justify the choice with overlap of audience, call topology, encryption/infra constraints, and historical group-call usage. Define the exact proxy metrics you will read (activation, 1/7/28-day retention of group callers, average group size, call minutes/user, failure rate) and how you will translate them into targets for this product while adjusting for platform differences.
4) Set or reject a participant cap: Should the feature enforce a maximum group size at launch? Provide a data-backed decision framework using historical distribution of multi-party interactions, infra capacity, call quality trade-offs, and moderation/safety. Propose an initial cap and a playbook to ramp it (or remove it) using guardrail metrics (setup success, join success, end-to-end latency, crash rate, abuse rate) and stop conditions.
5) Experiment design with network effects and contamination: Design the primary experiment to measure incremental value. Specify: unit of randomization (user, thread, ego-network cluster, or community), how you will prevent/mitigate interference, and the exposure policy when a treated user calls a control friend (block vs. allow with shadow treatment vs. invite-only gate). Quantify expected contamination given average degree d and treatment share p; propose a method (e.g., graph clustering or household-level hashing) to keep cross-edge spillovers under X%. Include primary KPIs (e.g., incremental daily callers, minutes, sender/recipient satisfaction proxy), guardrails (delivery latency, crash, spam/abuse), power analysis inputs (MDE, variance, sample size, test duration), and a staged ramp plan. Finally, explain how you would measure and interpret direct vs. indirect network effects (e.g., k-factor, changes in clustering coefficient) within this experiment.
Quick Answer: This question evaluates product analytics and experimentation competencies, including tie-strength and group-affinity measurement, event-level instrumentation design, proxy-product forecasting, participant-cap constraints and operational trade-offs, and experiment design that accounts for network effects and contamination.