Analyze Retention Data for Geo-Targeted Feature Launch
Quick Overview
DoorDash analytics prompt on evaluating a geo-targeted feature with retention curves, covering business case structure, metrics, limited-traffic success criteria, retention interpretation, geo experiments, and marketplace guardrails.
Analyze Retention Data for Geo-Targeted Feature Launch
Company: DoorDash
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
The company is deciding whether to launch a new geo-targeted feature; you have limited traffic data and a performance chart showing retention curves.
##### Question
Walk me through how you would structure this business case: what data you need, key metrics, and possible success criteria. Given this retention chart for test vs. control, what insights or hypotheses can you draw, and what next steps would you recommend?
##### Hints
Frame hypotheses, define measurable metrics, reason about trends, and propose follow-ups.
Quick Answer: DoorDash analytics prompt on evaluating a geo-targeted feature with retention curves, covering business case structure, metrics, limited-traffic success criteria, retention interpretation, geo experiments, and marketplace guardrails.
Business Case for a Geo-Targeted Feature With Retention Curves
The company is deciding whether to launch a new geo-targeted feature. You have limited traffic data and a performance chart showing retention curves for test versus control cohorts.
Assume the feature selectively surfaces local content or benefits by neighborhood or zone, and it could influence engagement, order frequency, revenue, and operational load in targeted geographies.
Constraints & Assumptions
Structure this as a business case and measurement plan.
Include minimal data needs, key metrics, success criteria, and retention-curve interpretation.
Limited traffic may make classical significance hard; propose practical decision criteria.
Address operational guardrails for a marketplace.
Clarifying Questions to Ask Guidance
What is the feature mechanism and target geography?
Is assignment randomized by user, zone, market, or rollout timing?
What does the retention chart show exactly: D1/D7/D28, weekly retention, or survival curve?
Are there concurrent promotions, seasonality, or operational changes?
What a Strong Answer Covers Guidance
Objective and hypotheses for retention, order frequency, revenue, and marketplace health.
Minimal data: exposure, treatment assignment, eligibility, geography, user cohorts, app version, retention outcomes, orders, AOV, contribution margin, and operational metrics.
Primary metrics such as D28 retention, weekly active rate, orders per user, and contribution margin per user.
Guardrails such as delivery time, cancellation, supply utilization, support contacts, and fairness across zones.
Success criteria under limited traffic: MDE, confidence intervals, Bayesian probability thresholds, directional consistency, and guardrail non-inferiority.
Retention-curve interpretation: early lift that fades, delayed lift, crossing curves, no effect, or negative effect, and what each implies.
Next steps: geo experiment, user-level randomization if feasible, diff-in-diff, synthetic control, cluster-robust SEs, follow-up tests, and operational readiness checks.
Follow-up Questions Guidance
How would you summarize a retention curve in one metric?
What if treatment retention is higher early but lower by day 28?
What would you do if traffic is too low for a powered test?
How would you detect spillovers across neighboring zones?