Plan an experiment to validate targeting impact

Q: Plan an experiment to validate targeting impact

This is a Analytics & Experimentation interview question from Stripe for Data Scientist roles. View the full question and solution on PracHub.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

You produced a ranked list of merchants predicted to adopt Subscription. Design an experiment to validate business impact of targeting them with a sales/email nudge. Constraints: randomize at merchant level among the top 50,000 scored merchants. Primary metric: conversion to first active Subscription within 30 days. Baseline conversion is 2.0% without outreach; minimal detectable uplift is +50% relative (to 3.0%). Costs: $5 outreach cost per merchant; expected 90-day gross margin from a converted merchant is $120. (a) Propose an RCT design (control vs. treatment split, stratification by score deciles and country), guardrails (refund rate, chargeback rate), and a staged ramp. (b) Compute the required per-arm sample size for 80% power and two-sided α=0.05 for proportions (show formula and numeric answer). (c) Given the costs and margin, compute the break-even precision threshold for targeting (ignore long-term spillovers). (d) Discuss contamination risks (multi-merchant customers), how to handle merchants already piloting Subscription, and how to attribute conversions when merchants receive multiple touches.

Plan an experiment to validate targeting impact

Comments (0)