## Onsite Project Review: Analyze and present an A/B test Before the onsite, you completed a take-home project analyzing an **A/B test** (you can assume typical product experimentation data: assignment, exposure, user events, and outcome metrics). During the onsite, you must present slides and answer deep-dive questions. ### What you should prepare 1. Summarize the experiment goal, design, and key assumptions. 2. Validate experiment integrity and data quality (what checks do you run?). 3. Estimate the treatment effect on pre-specified metrics. 4. Discuss interpretation and limitations (confounding risks, interference, multiple testing, seasonality). 5. Provide a clear ship/no-ship recommendation and next steps. ### Interviewer follow-ups to expect - What would you do if you see a sample ratio mismatch? - How do you pick primary vs guardrail metrics? - How do you handle many metrics or repeated looks at the data? - What if average impact is neutral but a segment improves a lot?