Prompt
You are a Data Scientist at an e-commerce platform (e.g., Shopify) launching a new App Store (a two-sided marketplace connecting merchants who install apps and developers who publish apps).
Task
Describe how you would measure whether the App Store launch is successful.
Your answer should include:
-
Success definition / goals
: what “success” means for the business and for key users.
-
Metrics framework
with:
-
A
North Star
metric (or 1–2 primary metrics)
-
Diagnostic
metrics (funnel + marketplace health)
-
Guardrail
metrics (avoid harming core product)
-
Segmentation
: which cohorts or segments you’d cut by (merchant size, app categories, geography, new vs existing, etc.).
-
Measurement strategy
:
-
How you would establish a baseline and evaluate lift (A/B test vs quasi-experiment)
-
How you would handle common issues (seasonality, cannibalization, delayed impact, selection bias)
-
What data you’d need
(key events/tables you would instrument or validate).
Output expectation
Provide a structured plan and explain tradeoffs. You may assume you can instrument events in the product and access transactional data.