Instagram Shopping: Sizing, Experiment Design, and Troubleshooting
Context
Instagram is launching an in‑app shopping feature. You are asked to:
-
Estimate the annual revenue opportunity from the feature.
-
Design an experiment to measure its impact on Gross Merchandise Value (GMV).
-
Troubleshoot a mid‑experiment conversion drop in the treatment group.
Assume: GMV is total dollar value of completed orders before returns; Instagram earns revenue via a take‑rate (fees/commission) on GMV. Focus on incremental impact versus status quo.
Tasks
-
Revenue sizing
-
Estimate annual revenue opportunity using a structured model (e.g., TAM × adoption × monetization/take‑rate). State assumptions clearly and provide a sensitivity range.
-
Experiment design to measure impact on GMV
-
Specify unit of randomization and exposure.
-
Define the primary metric, key secondary/funnel metrics, and guardrail metrics.
-
Describe a sample‑size/power approach suitable for heavy‑tailed GMV outcomes.
-
Mid‑experiment issue
-
Midway through the test, the treatment group’s conversion rate drops sharply. Outline a systematic, prioritized troubleshooting plan to diagnose and resolve the issue.