How do you design an A/B experiment?

Q: How do you design an A/B experiment?

This question evaluates competency in online experiment design, including hypothesis framing, metric selection and guardrails, unit of randomization and interference control, power/sample-size reasoning, and identification and mitigation of common biases.

Q: How do I approach Analytics interview questions?

Analytics questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics interviews.

Question

Loading...

You have an interview on your agenda titled “Experiment Design.”

You are asked to design an online experiment (A/B test) for a product change.

Describe, step-by-step:

The goal and hypothesis .
Primary/secondary metrics and guardrails.
Unit of randomization and how to avoid interference.
Sample size / power considerations (what inputs you need).
How you would handle biases (selection effects, novelty, seasonality, logging issues).
Decision rules, segmentation, and how you would interpret ambiguous results.

How do you design an A/B experiment?

Quick Overview

Solution

Comments (0)