Evaluate shopping tab pre- and post-launch

Quick Overview

This question evaluates a data scientist's competency in causal inference, experiment design, funnel metric instrumentation, revenue modeling, sensitivity analysis, and data‑quality triage within the Analytics & Experimentation domain.

Evaluate shopping tab pre- and post-launch

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Instagram plans a new Shopping tab but cannot observe off-app purchases. Design a rigorous end-to-end plan. Part A — Verify the hypothesis that users see purchase-inspiring posts on Instagram but buy on third-party sites: What measurable in-app signals would you use as proxies for off-app purchase intent (e.g., outbound merchant-link clicks, profile->bio link navigations, saves/shares with commerce hashtags, search-after-view patterns)? How would you quantify potential bias (e.g., selection, novelty, confounding from seasonality) and construct counterfactuals (e.g., synthetic controls, matched cohorts)? Part B — Pre-launch opportunity sizing: Assume DAU = 50M, 10% show intent weekly (as defined by a defensible in-app proxy), each intentor averages 0.2 purchases/week off-platform with AOV = $45, and a future platform take rate of 5%. Merchants are expected to spend ads/placement fees equal to $0.002 per impression for Shopping-tab surfaces with an estimated 6 impressions per intentor per week. Estimate weekly gross revenue impact by component (transaction fee vs. ads). What sensitivity analyses would you run and which 3 assumptions most dominate the estimate? Part C — Launch readout: You ship the tab and observe high click-through into merchant flows but purchases are far below forecast. Build a metric tree and investigation plan that (i) localizes drop-offs across the funnel (Discovery -> Product View -> Deep Link -> Cart -> Purchase), (ii) evaluates cannibalization of existing commerce surfaces (e.g., Marketplace) vs. true incrementality (design a geo or user-level holdout), (iii) distinguishes product/UX frictions from supply/quality issues (relevance, price, shipping), and (iv) rules out instrumentation/data-quality problems. Specify primary success metrics, guardrails (e.g., session length, feed CTR, ad revenue lift, complaint rate), required segmentation cuts, and what evidence would trigger a rollback.

Quick Answer: This question evaluates a data scientist's competency in causal inference, experiment design, funnel metric instrumentation, revenue modeling, sensitivity analysis, and data‑quality triage within the Analytics & Experimentation domain.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
5
0

Instagram Shopping Tab — Measuring Off‑App Purchases, Opportunity Sizing, and Launch Readout

Context

Instagram is planning a new Shopping tab. Users often discover products on Instagram but complete purchases on third‑party sites, which Instagram cannot directly observe. Design a rigorous, end‑to‑end plan to (A) verify the hypothesis that Instagram discovery drives off‑app purchases, (B) size the pre‑launch opportunity given assumptions, and (C) execute a launch readout that localizes funnel drop‑offs, quantifies incrementality vs. cannibalization, distinguishes UX vs. supply issues, and rules out data quality problems.

Part A — Verify the off‑app purchase hypothesis

  • What measurable in‑app signals would you use as proxies for off‑app purchase intent? Examples include outbound merchant‑link clicks, profile → bio link navigations, saves/shares with commerce hashtags, and search‑after‑view patterns.
  • How would you quantify potential bias (e.g., selection, novelty, confounding from seasonality) and construct counterfactuals (e.g., synthetic controls, matched cohorts)?

Part B — Pre‑launch opportunity sizing

  • Assumptions: DAU = 50M, 10% show intent weekly (by a defensible in‑app proxy), each intentor averages 0.2 purchases/week off‑platform, AOV = 45,futureplatformtakerate=545, future platform take rate = 5%. Merchants are expected to spend 0.002 per impression for Shopping‑tab surfaces with an estimated 6 impressions per intentor per week.
  • Estimate weekly gross revenue impact by component: transaction fee vs. ads.
  • What sensitivity analyses would you run, and which 3 assumptions most dominate the estimate?

Part C — Launch readout and investigation

You launch and observe high click‑through into merchant flows but purchases far below forecast. Build a metric tree and investigation plan that:

  1. Localizes drop‑offs across the funnel (Discovery → Product View → Deep Link → Cart → Purchase).
  2. Evaluates cannibalization of existing commerce surfaces (e.g., Marketplace) vs. true incrementality (design a geo or user‑level holdout).
  3. Distinguishes product/UX frictions from supply/quality issues (relevance, price, shipping).
  4. Rules out instrumentation/data‑quality problems. Specify primary success metrics, guardrails (e.g., session length, feed CTR, ad revenue lift, complaint rate), required segmentation cuts, and what evidence would trigger a rollback.
Loading comments...