Assess Success Criteria for Bike-Courier Delivery Launch
Company: DoorDash
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
##### Scenario
DoorDash plans to launch a bike-courier delivery option and wants to assess whether, where, and how to roll it out successfully.
##### Question
From DoorDash’s perspective, why launch a bike delivery option? What criteria would define success for the bike feature? How would you decide whether to launch in all markets or focus on urban/downtown areas? Which metrics (short-term, long-term, guardrail) would you track to judge success? How would you evaluate the feature’s impact, accounting for network effects? Describe how you would design and run an A/B test to measure the feature’s effectiveness. What data would you gather to assess downtown market suitability?
##### Hints
Consider marketplace balance for consumers, dashers, restaurants, efficiency, cost, delivery time, pilot-market choice, and robust experiment design.
Quick Answer: Evaluates marketplace analytics for launching a bike-courier delivery option in dense delivery markets. Strong answers define stakeholder value, success metrics, market selection, phased rollout, guardrails, and causal evaluation under network effects.
DoorDash plans to launch a bike-courier delivery option and wants to assess whether, where, and how to roll it out successfully.
As the data scientist leading the analysis, address the launch rationale, success criteria, market selection, metrics, evaluation strategy, and operational risks.
Constraints & Assumptions
Treat DoorDash as a multi-sided marketplace with customers, couriers, merchants, and platform economics.
Bike delivery is expected to be context-dependent by density, distance, weather, terrain, and infrastructure.
Consider both short-term operational metrics and long-term marketplace health.
Account for network effects and interference when designing evaluation.
Clarifying Questions to Ask Guidance
Which markets, order types, and distance bands are eligible for bike delivery?
Are bike couriers replacing car couriers, adding capacity, or both?
What are the primary launch goals: speed, cost, reliability, supply growth, sustainability, or coverage?
Are there safety, insurance, regulatory, or merchant constraints?
Part 1 - Launch Rationale and Beneficiaries
Explain why DoorDash might launch bike delivery and who benefits.
What This Part Should Cover Guidance
Describe customer benefits such as faster ETAs for short dense routes and potentially better reliability.
Describe courier benefits such as lower operating costs and access to urban earning opportunities.
Describe merchant and platform benefits such as faster handoffs, lower parking friction, higher supply density, or better unit economics.
Identify contexts where bikes may underperform.
Part 2 - Success Criteria and Metrics
Define what success means for customers, couriers, merchants, and the platform.
What This Part Should Cover Guidance
Include short-term metrics such as ETA accuracy, delivery time, assignment acceptance, cancellation, lateness, contact rate, and order defects.
Include long-term metrics such as retention, reorder rate, courier earnings, supply growth, market liquidity, and unit economics.
Include guardrails for safety, food quality, merchant wait time, weather reliability, and marketplace balance.
Segment metrics by distance, geography, time of day, merchant type, weather, and courier tenure.
Part 3 - Market Selection and Rollout Strategy
Decide between broad launch and focusing on dense urban or downtown areas.
What This Part Should Cover Guidance
Use density, route distance, traffic, parking difficulty, bike infrastructure, weather, hills, and merchant clustering.
Prefer phased rollout in high-fit markets before expanding.
Define eligibility rules for order assignment and courier supply onboarding.
Discuss operational readiness, support, safety policies, and training.
Part 4 - Evaluation Under Network Effects
Design an evaluation strategy that accounts for marketplace interference.
What This Part Should Cover Guidance
Explain why user-level randomization may be insufficient in a marketplace.
Consider geo-level experiments, switchbacks, market-level rollouts, or clustered randomization.
Track spillovers across courier supply, customer demand, merchant operations, and car-courier outcomes.
Use pre/post and difference-in-differences cautiously, with matched markets and robustness checks.
What a Strong Answer Covers Guidance
Connects launch decisions to both marketplace mechanics and causal measurement.
Balances customer speed, courier earnings, merchant reliability, safety, and economics.
Recommends a phased, segment-aware rollout rather than a blanket launch.
States how to act on mixed or segment-specific results.
Follow-up Questions Guidance
What would make you stop the rollout after a positive early ETA result?
How would you price or assign bike orders differently from car orders?
How would you measure whether bikes improve marketplace liquidity rather than merely shifting orders?