Design A/B Test to Evaluate Payment Method Impact

Q: Design A/B Test to Evaluate Payment Method Impact

This question evaluates a data scientist's skills in experimental design, causal inference, metric definition, statistical power estimation, and business trade-off analysis within the Analytics & Experimentation domain.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

A/B Experiment Design: New Payment Method Rollout

Context

You are evaluating whether to launch a new payment option across a country. Before launch, you can run an A/B experiment to estimate business impact. The payment option may change both conversion rate and average order value (AOV), and it may carry different payment processing fees and risk.

Tasks

Design an A/B test to assess whether the new payment method should be launched.
- Define hypotheses, experiment unit, eligibility, randomization, and exposure.
- Specify primary, secondary, and guardrail metrics.
- Outline instrumentation, sample size/power, duration, ramp plan, and analysis approach.
Suppose the experiment shows higher conversion rate but lower AOV. Explain how you would trade off these conflicting metrics.
Describe how you would communicate the recommendation and decision to the product team.

Design A/B Test to Evaluate Payment Method Impact

A/B Experiment Design: New Payment Method Rollout

Context

Tasks

Solution

Comments (0)