PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Meta

Design A/B Test to Evaluate Payment Method Impact

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's skills in experimental design, causal inference, metric definition, statistical power estimation, and business trade-off analysis within the Analytics & Experimentation domain.

  • medium
  • Meta
  • Analytics & Experimentation
  • Data Scientist

Design A/B Test to Evaluate Payment Method Impact

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Onsite

##### Scenario A countrywide rollout of a new payment option is being considered. You can run an A/B test to evaluate business impact before launch. ##### Question How would you design an A/B experiment to assess whether the new payment method should be launched? Suppose conversion rate rises while average order value drops. How do you trade off these conflicting metrics, and how would you communicate the recommendation to the product team? ##### Hints Define primary/guardrail metrics, compute overall revenue or LTV impact, discuss statistical significance and business context.

Quick Answer: This question evaluates a data scientist's skills in experimental design, causal inference, metric definition, statistical power estimation, and business trade-off analysis within the Analytics & Experimentation domain.

Related Interview Questions

  • Measure scheduled posts feature success - Meta (medium)
  • Estimate ads ranking revenue impact - Meta (medium)
  • How should you evaluate unconnected content? - Meta (medium)
  • Should WhatsApp launch group calls? - Meta (medium)
  • How would you grow Meta products? - Meta (medium)
Meta logo
Meta
Aug 4, 2025, 10:55 AM
Data Scientist
Onsite
Analytics & Experimentation
3
0

A/B Experiment Design: New Payment Method Rollout

Context

You are evaluating whether to launch a new payment option across a country. Before launch, you can run an A/B experiment to estimate business impact. The payment option may change both conversion rate and average order value (AOV), and it may carry different payment processing fees and risk.

Tasks

  1. Design an A/B test to assess whether the new payment method should be launched.
    • Define hypotheses, experiment unit, eligibility, randomization, and exposure.
    • Specify primary, secondary, and guardrail metrics.
    • Outline instrumentation, sample size/power, duration, ramp plan, and analysis approach.
  2. Suppose the experiment shows higher conversion rate but lower AOV. Explain how you would trade off these conflicting metrics.
  3. Describe how you would communicate the recommendation and decision to the product team.

Solution

Show

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.