Measure Ultrafast Delivery's Impact Using Synthetic Control Method
Overview
This question evaluates a data scientist’s competency in causal inference and experimental analytics, focusing on panel-data methods and control-group selection for measuring the impact of a city-level product launch.
Scenario
Instacart launched Ultrafast Delivery in Miami two months ago and wants to measure its causal impact on user order volume.
Assume you have panel data at the daily or weekly level for multiple geographies (cities/ZIPs), including Miami and a set of non-launched geographies, with pre- and post-launch history. You also have covariates like baseline demand, seasonality, retailer mix, promos, and weather.
Task
Design an approach to estimate the feature’s causal impact on orders and describe how you would select an appropriate control geography.
Requirements
-
Control Geography Selection
- How would you choose and validate a control geography (or set of geographies)?
-
Method Mechanics
- Describe the mechanics of your chosen causal method (e.g., Difference-in-Differences, Synthetic Control, Propensity-Score Matching). Be explicit about identification assumptions and how you’ll check parallel trends.
-
Linear Mixed-Effects Variant
- If you choose a linear mixed-effects model, specify which variables would be fixed versus random.
-
Robustness and Validation
- Discuss parallel-trend checks, placebo tests, sensitivity analyses, and how to calculate the impact metric and uncertainty.
Solution
Login required