Investigate marketplace metrics and experiment rollout

You are interviewing for a Senior Data Scientist role at a two-sided marketplace like Instacart, where customers place delivery orders and shoppers choose whether to accept and fulfill them.

Answer the following related interview questions:

1. A core marketplace metric has declined over the past two weeks. Describe a structured approach to determine whether the decline is real and to identify likely causes. Your plan should consider data-quality issues, seasonality, pricing or product changes, supply-demand balance, user/shopper/merchant segments, and external factors such as competitors or weather.
2. The company wants to launch a new pricing model that pays shoppers more during rush hours to incentivize them to pick up more orders. Because shopper behavior and customer demand interact within each local market, a standard user-level A/B test may suffer from interference and network effects. Design an experiment. Specify the unit of randomization, how you would construct matched or lookalike markets, what pre-period covariates and balance checks you would use, the primary success metrics, the guardrail metrics, and how you would analyze the results while accounting for spillovers and seasonality.
3. Suppose the experiment has finished. The pre-declared north-star metric is profit per order. The treatment increases average order volume, but profit per order decreases. Should the company roll out the pricing model? Explain how you would reason about a negative north-star metric alongside positive secondary metrics, what follow-up analyses are valid, and when a no-launch recommendation is still the correct decision.
4. A dashboard shows that D14 retention fell sharply for the most recent week, while older cohorts look stable. Explain how you would determine whether this is a real product problem or a false alarm caused by immature cohorts, delayed event ingestion, or metric-definition issues. Be explicit about cohorting, right-censoring, and what the correct monitoring view should be.