Choosing between design variants with A/B tests in partnership with a designer

Describe how you and a designer have used A/B tests to choose between versions of a design. How did the two of you get from early concepts to a decision? ```hint Find where design meets the experiment Think about the points where the designer's work and the test meet: which versions earn live traffic, what counts as winning, and what happens when the result surprises one of you. ``` ### Clarifying Questions - Should the answer center on one experiment, or on how you usually run design tests with designers? - Were the versions small changes (copy, layout, a single control) or substantially different flows? - Does the interviewer want detail on the experiment setup (metrics, sample size, duration), or mainly on the collaboration? ### What a Strong Answer Covers - How candidate versions were chosen and narrowed, including what was settled with qualitative research before spending traffic. - A hypothesis, primary metric and guardrails agreed with the designer before launch. - Sound experiment basics: enough traffic for the number of variants, a run long enough for weekly patterns and novelty, and changes isolated enough to learn from. - How results were read and acted on together, including a losing favorite or an inconclusive result. - What the partnership produced beyond one launch: shared learnings and the next round of iterations. ### Follow-up Questions - The version the designer was most invested in lost. How did you handle that conversation? - The test showed no significant difference between versions. What did you decide, and on what basis? - When would you choose not to A/B test a design change at all?