Describe an A/B test you owned or analyzed. Explain the original decision, your role, the primary metric, and the estimated incremental lift. Then discuss a follow-up test, the teams involved, and a trade-off that led you not to launch one option even if part of the result looked promising. ### Constraints & Assumptions - Use a real example you can defend under detailed follow-up; anonymize confidential values when necessary. - Distinguish incremental causal lift from raw before-and-after movement. - State uncertainty and guardrails rather than reporting a single favorable number. - Be precise about what you personally decided, implemented, or influenced. ### Clarifying Questions to Ask - Should the answer focus on experiment design, stakeholder leadership, or both? - Is an example with a no-launch decision preferred? - May commercially sensitive values be expressed as indexed or rounded figures? ```hint Prepare one story for a deep drill-down Connect every follow-up to the same decision: why the test existed, what changed, how lift was identified, and why the final action followed from the evidence. ``` ### What a Strong Answer Covers - A clear hypothesis, randomization unit, primary metric, and success rule. - Incremental lift with denominator, comparison, uncertainty, and measurement window. - A follow-up experiment motivated by the first result. - Cross-functional ownership and a concrete disagreement or constraint. - A trade-off explaining why shipping was not automatically the right outcome. ### Follow-up Questions - How do you know the reported lift was incremental rather than seasonal? - Which guardrail came closest to changing the decision? - What did the follow-up test isolate that the first test could not? - What evidence would have changed the no-launch decision?