Determine User Need for In-App Video Call Feature

Quick Overview

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Determine User Need for In-App Video Call Feature states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Determine User Need for In-App Video Call Feature

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

##### Scenario Company is considering launching a new in-app Video Call feature and has full historical user and call data available. ##### Question Using the available data, how would you determine whether users need an in-app Video Call feature? 2. After launch, what success and guard-rail metrics would you track to evaluate feature quality? 3. Describe step-by-step how you would design and run an A/B test for this feature. 4. What are the engineering pros and cons of enforcing a hard hang-off (automatic call termination) policy, and how might that affect experiment results? ##### Hints Think funnel, engagement, retention, power-user segmentation, experiment unit, sample size, run duration, and potential user experience trade-offs.

Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Determine User Need for In-App Video Call Feature states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Aug 4, 2025, 10:55 AM
hardData ScientistTechnical ScreenAnalytics & Experimentation
2
0

Determine User Need for In-App Video Call Feature

Scenario

A consumer messaging app is considering launching an in-app Video Call feature. You have access to full historical user and call data (e.g., messaging, audio call usage, device/network attributes, and user engagement/retention).

Questions

  1. Using the available data, how would you determine whether users need an in-app Video Call feature before launch?
  2. After launch, what success metrics and guard-rail metrics would you track to evaluate feature quality?
  3. Describe step-by-step how you would design and run an A/B test for this feature.
  4. What are the engineering pros and cons of enforcing a hard hang-off (automatic call termination) policy, and how might that affect experiment results?

Hint: Think funnel, engagement, retention, power-user segmentation, experiment unit, sample size, run duration, and potential user experience trade-offs.

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
  • State assumptions about instrumentation, randomization, sample size, and data quality.
  • Separate descriptive analysis from causal claims.

What a Strong Answer Covers Guidance

  • A metric framework with primary, guardrail, and diagnostic metrics.
  • A credible analysis or experiment design with clear assumptions and bias checks.
  • SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
  • An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions Guidance

  • What sanity checks would you run before trusting the result?
  • How would you handle novelty effects, seasonality, or selection bias?
  • What decision would you make if metrics disagree?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation