Design an experiment for pricing page redesign

Q: Design an experiment for pricing page redesign

This question evaluates a candidate's competency in experiment design, causal inference, impact sizing, metric framework development, and statistical analysis in the context of a pricing page redesign that adds an annual subscription option.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

A product team is redesigning a pricing tier page. Historically the page only offered a monthly plan; the redesign adds an annual plan option.

You are the Data Scientist partner for the launch.

Questions

Impact sizing: What are the key ways this change could impact the business (positive and negative), and how would you estimate the expected impact magnitude before running anything?
Metrics: Propose a metric framework:
- Primary success metric(s)
- Diagnostic metrics (to understand why it moved)
- Guardrail metrics (to prevent harm)
Experiment design & ship decision: Describe how you would design the experiment and make a ship / no-ship decision, including:
- Unit of randomization and key segments to monitor
- Sample size / power approach (MDE) and duration considerations
- How you would analyze results (e.g., confidence intervals) and handle pitfalls (SRM, novelty, repeated exposure)
No experiment possible: If you cannot run an A/B test (policy, engineering constraints, or all-users launch), how would you measure impact as credibly as possible?

Design an experiment for pricing page redesign

Questions

Solution

Comments (0)

Design an experiment for pricing page redesign

Overview

Questions

Solution

Comments (0)