Size opportunity and prioritize experiments

Quick Overview

This question evaluates proficiency in market sizing, funnel-based revenue modeling, A/B test power and minimum detectable effect analysis, experiment design and guardrails, cannibalization risk assessment, and measurement of short‑ vs long‑term trade-offs within the Analytics & Experimentation domain for a Data Scientist role.

Size opportunity and prioritize experiments

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Before investing engineering time, quantify whether a new e-commerce product line is worth testing. 1) Do top-down TAM/SAM/SOM sizing and bottom-up sizing from existing traffic and conversion, explicitly estimating what percentage of users (>20% vs ~5%) will be affected; 2) compute expected incremental revenue and the minimum detectable uplift that makes the test economically viable (include fixed costs and opportunity costs of traffic); 3) propose a test plan: targeting, ramp, KPIs, counter-metrics, and a go/no-go bar; 4) detail how you’d de-risk cannibalization of existing categories and identify leading indicators before revenue shows up; 5) specify what you’d do if short-term engagement drops but long-term retention is likely to improve.

Quick Answer: This question evaluates proficiency in market sizing, funnel-based revenue modeling, A/B test power and minimum detectable effect analysis, experiment design and guardrails, cannibalization risk assessment, and measurement of short‑ vs long‑term trade-offs within the Analytics & Experimentation domain for a Data Scientist role.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
6
0

New E‑commerce Product Line: Pre‑Investment Quantification and Test Plan

You are evaluating whether to invest engineering and operational resources to launch a new e‑commerce product line on an existing consumer platform. Assume you can target by geo/device, run A/B tests for 2–4 weeks, and have access to historical traffic, conversion, AOV, and margin data.

Provide the following:

  1. Market sizing
    • Top‑down: Estimate TAM, SAM, and SOM for the category; state assumptions.
    • Bottom‑up: Size impact using current traffic and funnel metrics. Explicitly model two exposure scenarios: a broad surface affecting >20% of users and a niche surface affecting ~5%.
  2. Economics and detectability
    • Compute expected incremental revenue (or contribution margin) from launch over a 12‑month horizon.
    • Include fixed build/operational costs and the opportunity cost of test traffic.
    • Compute the minimum detectable uplift (MDE) for a 2‑week A/B test and compare it to the economically required uplift to be viable.
  3. Experiment design
    • Propose a test plan: targeting, ramp schedule, primary KPIs, counter‑metrics/guardrails, and a clear go/no‑go bar tied to business value.
  4. Cannibalization risk
    • Show how you would de‑risk cannibalization of existing categories and define leading indicators you’d monitor before revenue impacts are visible.
  5. Short‑term vs long‑term trade‑offs
    • Specify what you’d do if short‑term engagement drops but long‑term retention is likely to improve. Include measurement plan and decision rules.
Loading comments...