How would you investigate a metric decline?
Company: Instacart
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Technical Screen
## Scenario
You are a Data Scientist supporting a consumer marketplace product. A key business metric (e.g., orders/day, conversion rate, revenue, profit, retention) has **declined noticeably** over the last 1–2 weeks.
## Task
Describe a structured approach to investigate the decline.
### Requirements
In your answer, cover:
1. **Clarify the metric**
- Exact definition, numerator/denominator, filters, attribution logic.
- Timezone, event time vs processing time, and whether the metric is cohort-based.
2. **Detect whether it’s real vs measurement**
- Data pipeline issues, logging changes, bot/traffic anomalies, backfills.
3. **Localize the change**
- When did it start? step vs trend? aligned to releases/experiments?
4. **Decompose and segment**
- Break down by funnel stage and by key segments (new vs returning, geo, device, acquisition channel, product surfaces, supply vs demand side, etc.).
5. **Consider external and seasonal factors**
- Seasonality/holidays, competitor actions, macro events, weather, outages.
6. **Form hypotheses and propose follow-ups**
- Additional analyses, dashboards, and (if appropriate) experiments or mitigations.
### Output
Explain the investigation plan and what you would do in the first day vs first week.
Quick Answer: This question evaluates a data scientist's competency in diagnostic analytics, metric instrumentation awareness, causal reasoning, and segmentation-driven problem localization for consumer marketplace KPIs.