Design an experiment to evaluate an onboarding progress bar
Company: Intuit
Role: Product Analyst
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Onsite
## Scenario
In a QuickBooks-like onboarding flow, the product team adds a **progress bar**.
The onboarding has 4 steps:
1. Onboard
2. Send payment
3. Set up bank account
4. File the tax
## Question
How would you evaluate whether the progress bar **helps**?
Your answer should include:
- Hypothesis and what “helps” means
- Primary metric + diagnostic metrics + guardrails (with tradeoffs)
- Experiment design (randomization unit, sample size/power considerations at a high level, duration)
- Key pitfalls (novelty effects, instrumentation, segment heterogeneity) and how you’d address them
Quick Answer: This question evaluates a candidate's competency in experiment design, A/B testing, metric selection and trade-offs, causal inference, and product analytics for measuring the impact of a UI element like a progress bar.