##### Scenario A/B testing and product analytics discussions ##### Question Walk me through how you would design, run, and analyze an A/B test for a new checkout feature. Case study: PayPal plans to launch a cashback program. How would you evaluate its success, what metrics would you track, and what data would you need? ##### Hints State hypothesis, choose primary & guardrail metrics, randomization, sample-size, test duration, segmentation, analysis plan.

This question evaluates experimental design, causal inference, statistical analysis, instrumentation, and product analytics competencies for running and interpreting A/B tests on promotional checkout features.

How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at PayPal.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at PayPal during technical interviews.

Design and Analyze A/B Test for Cashback Program

A/B Test Design: Checkout Cashback Program (PayPal)

Scenario

PayPal plans to launch a checkout cashback program (e.g., "Get 1–5% back when you pay with PayPal"). The goal is to evaluate whether offering cashback at checkout improves key business outcomes while remaining cost-effective and safe.

Task

Design, run, and analyze an A/B test to evaluate the cashback program.

Please cover

Hypothesis and success criteria.
Experiment design:
- Population and eligibility
- Unit of randomization and variants
- Exposure and assignment rules
- Sample size and test duration
Metrics:
- Primary metric(s)
- Guardrail/safety metrics
- Secondary/diagnostic metrics
Data needed and instrumentation.
Analysis plan and statistical methods.
Segmentation and heterogeneity of effects.
Risks, biases, and mitigations (e.g., fraud, interference, novelty).

You may assume the feature is shown at checkout to eligible users and credits cashback after successful payment.

A/B Test Design: Checkout Cashback Program (PayPal)

Scenario

Task

Design, run, and analyze an A/B test to evaluate the cashback program.

Please cover

Hypothesis and success criteria.
Experiment design:
- Population and eligibility
- Unit of randomization and variants
- Exposure and assignment rules
- Sample size and test duration
Metrics:
- Primary metric(s)
- Guardrail/safety metrics
- Secondary/diagnostic metrics
Data needed and instrumentation.
Analysis plan and statistical methods.
Segmentation and heterogeneity of effects.
Risks, biases, and mitigations (e.g., fraud, interference, novelty).

You may assume the feature is shown at checkout to eligible users and credits cashback after successful payment.

Design and Analyze A/B Test for Cashback Program

Quick Overview

A/B Test Design: Checkout Cashback Program (PayPal)

Scenario

Task

Please cover

Solution

Comments (0)

Design and Analyze A/B Test for Cashback Program

Quick Overview

A/B Test Design: Checkout Cashback Program (PayPal)

Scenario

Task

Please cover

Solution

Comments (0)