Should you roll out if NSM decreases?

## Scenario You ran an experiment. The **north star metric (NSM)** is **profit per order**. ### Observed results - **Average order volume increased** in treatment vs control. - **Profit per order decreased** (statistically and/or practically meaningfully). ## Task Should you roll out the change? Explain your decision process. ### Requirements In your answer, cover: - Why optimizing the NSM matters vs secondary metrics. - What additional checks you would run (segment analysis, guardrails, novelty effects, heterogeneous treatment effects). - When (if ever) you would still consider launching (e.g., if total profit increases, long-term effects, strategic goals). - A clear final recommendation and next steps.