Design an experiment to evaluate an onboarding progress bar

Q: Design an experiment to evaluate an onboarding progress bar

This question evaluates a candidate's competency in experiment design, A/B testing, metric selection and trade-offs, causal inference, and product analytics for measuring the impact of a UI element like a progress bar.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Scenario

In a QuickBooks-like onboarding flow, the product team adds a progress bar.

The onboarding has 4 steps:

Onboard
Send payment
Set up bank account
File the tax

Question

How would you evaluate whether the progress bar helps?

Your answer should include:

Hypothesis and what “helps” means
Primary metric + diagnostic metrics + guardrails (with tradeoffs)
Experiment design (randomization unit, sample size/power considerations at a high level, duration)
Key pitfalls (novelty effects, instrumentation, segment heterogeneity) and how you’d address them

Design an experiment to evaluate an onboarding progress bar

Scenario

Question

Solution

Comments (0)

Design an experiment to evaluate an onboarding progress bar

Overview

Scenario

Question

Solution

Comments (0)