Measure Impact of Merchant Variety on Consumer Experience
Overview
This question evaluates a data scientist's ability to define supply- and consumer-level variety metrics and to design causal experiments that quantify how expanding merchant selection impacts consumer experience and marketplace health.
DoorDash
Aug 4, 2025, 10:55 AM
Data Scientist
Onsite
Analytics & Experimentation
15
0
Scenario
A two‑sided food delivery marketplace wants to understand the impact of expanding merchant selection (e.g., onboarding more restaurants, expanding delivery radius, surfacing more diverse options) on consumer experience and marketplace health.
Task
- Define success metrics that capture merchant variety at both the supply (merchant/market) and consumer levels.
- Propose an experiment (or alternative causal design) to quantify the effect of increasing selection on user outcomes, with attention to unit of randomization, guardrails, and heterogeneous effects.
Questions
- What metrics would you use to measure merchant variety at the merchant/market level and at the consumer level?
- How would you design and analyze an A/B test (or alternative) to measure the causal effect of increasing selection?
Hints
- Consider diversity indices (Shannon entropy, Simpson, HHI), conversion, retention.
- Specify unit of randomization, guardrail metrics, and heterogeneous effects analyses.
Solution
Login required
Show