Estimate Instagram Shopping Feature's Revenue and Test Impact

Quick Overview

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Estimate Instagram Shopping Feature's Revenue and Test Impact states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Estimate Instagram Shopping Feature's Revenue and Test Impact

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

##### Scenario Instagram plans to launch an in-app shopping feature and needs an analytical framework to size, test, and troubleshoot it. ##### Question Estimate the annual revenue opportunity of introducing shopping on Instagram. Design an A/B test to measure the feature’s impact on GMV—state unit of randomization, primary and guardrail metrics, and sample-size approach. Mid-experiment, the treatment group’s conversion rate drops sharply. Outline a systematic troubleshooting plan. ##### Hints Break sizing into TAM × adoption × take-rate, clarify success/failure metrics, and examine data, logging, and external factors when debugging.

Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Estimate Instagram Shopping Feature's Revenue and Test Impact states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Aug 4, 2025, 10:55 AM
hardData ScientistOnsiteAnalytics & Experimentation
3
0

Estimate Instagram Shopping Feature's Revenue and Test Impact

Instagram Shopping: Sizing, Experiment Design, and Troubleshooting

Context

Instagram is launching an in‑app shopping feature. You are asked to:

  • Estimate the annual revenue opportunity from the feature.
  • Design an experiment to measure its impact on Gross Merchandise Value (GMV).
  • Troubleshoot a mid‑experiment conversion drop in the treatment group.

Assume: GMV is total dollar value of completed orders before returns; Instagram earns revenue via a take‑rate (fees/commission) on GMV. Focus on incremental impact versus status quo.

Tasks

  1. Revenue sizing
  • Estimate annual revenue opportunity using a structured model (e.g., TAM × adoption × monetization/take‑rate). State assumptions clearly and provide a sensitivity range.
  1. Experiment design to measure impact on GMV
  • Specify unit of randomization and exposure.
  • Define the primary metric, key secondary/funnel metrics, and guardrail metrics.
  • Describe a sample‑size/power approach suitable for heavy‑tailed GMV outcomes.
  1. Mid‑experiment issue
  • Midway through the test, the treatment group’s conversion rate drops sharply. Outline a systematic, prioritized troubleshooting plan to diagnose and resolve the issue.

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
  • State assumptions about instrumentation, randomization, sample size, and data quality.
  • Separate descriptive analysis from causal claims.

What a Strong Answer Covers Guidance

  • A metric framework with primary, guardrail, and diagnostic metrics.
  • A credible analysis or experiment design with clear assumptions and bias checks.
  • SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
  • An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions Guidance

  • What sanity checks would you run before trusting the result?
  • How would you handle novelty effects, seasonality, or selection bias?
  • What decision would you make if metrics disagree?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation