Evaluate Miami Ultrafast impact on orders
Overview
This question evaluates competency in causal inference and panel-data experimentation, including control selection, pre-trend validation, difference-in-differences and event-study estimation, mixed-effects modeling, interference and seasonality handling, staggered-adoption inference, and reporting of treatment effects.
Instacart launched an Ultrafast delivery feature in Miami two months ago. You have weekly orders per geo for the past 52 weeks; Miami is treated from week t0 onward, other geos have not launched yet. Evaluate the causal impact on user orders. Address: (1) Control selection: propose two methods (e.g., matched-markets using propensity scores on pre-period covariates and outcomes; synthetic control minimizing pre-period RMSE) and how you’d validate the choice. (2) Assumptions: state the parallel-trends assumption; design and interpret pre-trend tests and an event-study plot; specify minimum pre-period length. (3) Estimation: write the difference-in-differences or event-study regression with geo and week fixed effects, clustering SEs appropriately; if using a linear mixed-effects model, identify which effects are fixed (e.g., treatment×post, week effects) and which are random (e.g., geo intercepts and/or slopes), and justify. (4) Threats: handle interference/spillovers (e.g., exclude adjacent buffer geos), anticipation, and seasonality. (5) Staggered adoption: if other geos later adopt, name an estimator robust to staggered timing (e.g., Sun–Abraham or Callaway–Sant’Anna) and outline implementation. (6) Reporting: define the primary metric (e.g., orders per active user), compute ATT with a 95% CI, and interpret a hypothetical +3% lift.