Evaluate Account-Partner Onboarding with Success Metrics
Overview
This question evaluates a data scientist's competency in defining core success metrics, estimating causal impact from observational data, constructing propensity-score or synthetic-control comparison groups, and modeling acquisition-cost versus incremental-revenue relationships.
Evaluating an Account-Partner Merchant Onboarding Program
Context
You operate a two‑sided marketplace that onboards new merchants via two channels: (1) account partners (outbound sales) and (2) organic/self-serve. Leadership wants to evaluate how well the account-partner program works in terms of business impact, efficiency, and scalability. Randomized experiments are not currently available.
Questions
(a) Which core success metrics would you report to evaluate account-partner onboarding performance?
(b) Without randomized experiments, how would you estimate causal impact (e.g., using difference‑in‑differences)?
(c) If account partners mainly sign seafood restaurants, how would you build a propensity‑score or synthetic‑control comparison group, and which merchant attributes would you match on?
(d) How would you model the relationship between acquisition cost and incremental revenue, and why might a logistic rather than linear curve be appropriate?
Solution
Login required