Evaluate Account-Partner Onboarding with Success Metrics

Quick Overview

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate Account-Partner Onboarding with Success Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Evaluate Account-Partner Onboarding with Success Metrics

Company: DoorDash

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

##### Scenario DoorDash's account-partner team acquires new merchants onto the marketplace, and leadership wants to quantify how well that program works. Because there is no randomized experiment, you must rely on observational data, and the partners tend to concentrate on a single cuisine (e.g., seafood restaurants). ##### Question 1. Which core success metrics would you report to evaluate account-partner onboarding performance? 2. Without randomized experiments, how would you estimate the causal impact of the program (e.g., using difference-in-differences)? 3. If account partners mainly sign seafood restaurants, how would you build a propensity-score or synthetic-control comparison group, and which merchant attributes would you match on? 4. How would you model the relationship between acquisition cost (or spend) and incremental revenue, and why might a logistic (S-shaped) curve be more appropriate than a linear one? ##### Hints Think causal inference: parallel-trend checks, matching, and merchant-level covariates (location, cuisine, size, tenure). For the spend-response curve, reason about diminishing returns and bounded (TAM-constrained) outcomes.

Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate Account-Partner Onboarding with Success Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Analytics & Experimentation/DoorDash
DoorDash logo
DoorDash
Aug 4, 2025, 10:55 AM
hardData ScientistOnsiteAnalytics & Experimentation
88
0

Evaluate Account-Partner Onboarding with Success Metrics

Scenario

DoorDash's account-partner team acquires new merchants onto the marketplace, and leadership wants to quantify how well that program works. Because there is no randomized experiment, you must rely on observational data, and the partners tend to concentrate on a single cuisine (e.g., seafood restaurants).

Question
  1. Which core success metrics would you report to evaluate account-partner onboarding performance?
  2. Without randomized experiments, how would you estimate the causal impact of the program (e.g., using difference-in-differences)?
  3. If account partners mainly sign seafood restaurants, how would you build a propensity-score or synthetic-control comparison group, and which merchant attributes would you match on?
  4. How would you model the relationship between acquisition cost (or spend) and incremental revenue, and why might a logistic (S-shaped) curve be more appropriate than a linear one?
Hints

Think causal inference: parallel-trend checks, matching, and merchant-level covariates (location, cuisine, size, tenure). For the spend-response curve, reason about diminishing returns and bounded (TAM-constrained) outcomes.

Clarifying Questions to Ask Guidance

  • Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
  • State assumptions about instrumentation, randomization, sample size, and data quality.
  • Separate descriptive analysis from causal claims.

What a Strong Answer Covers Guidance

  • A metric framework with primary, guardrail, and diagnostic metrics.
  • A credible analysis or experiment design with clear assumptions and bias checks.
  • SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
  • An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions Guidance

  • What sanity checks would you run before trusting the result?
  • How would you handle novelty effects, seasonality, or selection bias?
  • What decision would you make if metrics disagree?
Loading comments...