Design and Interpret an A/B Test
Company: Instacart
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Onsite
Overview: This question evaluates a data scientist's competency in experimental design and applied statistics, encompassing power and sample-size calculations, translating user-level sample requirements into experiment duration given traffic and exposure windows, awareness of seasonality and novelty effects, optional stopping and Type I error control, and correct interpretation of p-values and confidence intervals. It is commonly asked in Analytics & Experimentation interviews because employers need assurance that practitioners can design and interpret robust A/B tests; the domain is experimental design and statistical inference and the task requires both conceptual understanding of statistical principles and practical application to traffic, timing, and analysis constraints.