Evaluate Impact of $1 Fee on Fast-Food Profitability
Experiment Design: $1 Delivery-Fee Surcharge on Unprofitable Restaurants
Scenario
About 10% of fast-food restaurants on the platform are unprofitable. Product wants to add a $1 delivery-fee surcharge to orders from those restaurants to cover the deficit.
Task
Design an experiment to evaluate the impact of the $1 fee on:
-
Restaurant-level profitability
-
Order volume (conversion and frequency)
-
Customer satisfaction
Include:
-
Experimental unit and randomization plan (address marketplace spillovers/interference).
-
Primary/secondary metrics and guardrails, with clear hypotheses.
-
Sample size and duration guidance (state assumptions if needed).
-
Analysis plan (e.g., ITT vs. TOT, variance reduction, decomposition of effects).
-
Concrete techniques to increase statistical power (e.g., stratified randomization, CUPED, covariate blocking, geo-testing, longer horizon), and when to use each.
Assume the $1 fee is only applied to currently unprofitable restaurants. If needed, make minimal additional assumptions explicit.
Constraints & Assumptions
-
Preserve the scope, facts, inputs, and requested outputs from the prompt above.
-
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
-
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.
Clarifying Questions to Ask
-
Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
-
State assumptions about instrumentation, randomization, sample size, and data quality.
-
Separate descriptive analysis from causal claims.
What a Strong Answer Covers
-
A metric framework with primary, guardrail, and diagnostic metrics.
-
A credible analysis or experiment design with clear assumptions and bias checks.
-
SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
-
An actionable recommendation that explains trade-offs and next steps.
Follow-up Questions
-
What sanity checks would you run before trusting the result?
-
How would you handle novelty effects, seasonality, or selection bias?
-
What decision would you make if metrics disagree?