Design a pricing experiment with network effects

Quick Overview

This question evaluates experimental design and causal inference competencies for a two-sided marketplace pricing change, including handling network effects/interference, choice of unit of randomization and treatment markets, metric selection and diagnostics, power/MDE estimation, and operational risk assessment within the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked to assess practical application of randomized and quasi-experimental methods across heterogeneous geographies and operational constraints, requiring both hands-on experimental design skills and conceptual understanding of bias, spillovers, and diagnostic metrics.

Design a pricing experiment with network effects

Company: Instacart

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: easy

Interview Round: Technical Screen

## Scenario You want to launch a **new pricing model** that incentivizes shoppers to place/pick up more orders during **rush hours** in a two-sided marketplace (supply and demand interact). You suspect **network effects / interference**: changing prices for some users may affect availability, ETAs, or acceptance rates for others. ## Task Design an experiment to evaluate the new pricing model. ### Constraints - A standard user-level A/B test may be invalid due to **spillovers** (interference) across users. - The marketplace has heterogeneous geographies with different baselines. ### Requirements Your design should include: 1. **Unit of randomization** and why (e.g., geo/market-level). 2. How you will choose **treatment/control markets** (e.g., matched pairs / lookalikes). 3. **Primary metric** (north star) and a set of diagnostic + guardrail metrics. 4. How you handle **bias/confounding** (seasonality, pre-trends, market differences). 5. Ramp plan, duration, and how you’ll estimate power/MDE at a high level. 6. Risks: spillovers across nearby markets, partial compliance, concurrent changes. ### Output Provide a clear experimental plan and analysis approach (e.g., difference-in-differences).

Quick Answer: This question evaluates experimental design and causal inference competencies for a two-sided marketplace pricing change, including handling network effects/interference, choice of unit of randomization and treatment markets, metric selection and diagnostics, power/MDE estimation, and operational risk assessment within the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked to assess practical application of randomized and quasi-experimental methods across heterogeneous geographies and operational constraints, requiring both hands-on experimental design skills and conceptual understanding of bias, spillovers, and diagnostic metrics.

|Home/Analytics & Experimentation/Instacart
Instacart logo
Instacart
Feb 6, 2026, 12:33 PM
easyData ScientistTechnical ScreenAnalytics & Experimentation
26
0

Scenario

You want to launch a new pricing model that incentivizes shoppers to place/pick up more orders during rush hours in a two-sided marketplace (supply and demand interact). You suspect network effects / interference: changing prices for some users may affect availability, ETAs, or acceptance rates for others.

Task

Design an experiment to evaluate the new pricing model.

Constraints

  • A standard user-level A/B test may be invalid due to spillovers (interference) across users.
  • The marketplace has heterogeneous geographies with different baselines.

Requirements

Your design should include:

  1. Unit of randomization and why (e.g., geo/market-level).
  2. How you will choose treatment/control markets (e.g., matched pairs / lookalikes).
  3. Primary metric (north star) and a set of diagnostic + guardrail metrics.
  4. How you handle bias/confounding (seasonality, pre-trends, market differences).
  5. Ramp plan, duration, and how you’ll estimate power/MDE at a high level.
  6. Risks: spillovers across nearby markets, partial compliance, concurrent changes.

Output

Provide a clear experimental plan and analysis approach (e.g., difference-in-differences).

Loading comments...