Experiment design: measuring whether a gift card incentive drives Prime app adoption

Amazon wants to use a gift card promotion to get Prime members to download the Prime mobile app. Design the test you would run to measure whether the promotion succeeds. Cover which members you would include and how you would assign them, what you would measure, how long the test would run and how you would decide whether to roll the promotion out. ```hint Not every install is earned Some members would have downloaded the app with no offer at all, and an app that is opened once to spend the gift card and then forgotten is worth little. Think about what comparison and what later behavior would show the promotion itself made a difference. ``` ### Clarifying Questions - Which app is meant: Prime Video, the main Amazon shopping app, or another Prime app? - What must a member do to earn the gift card (install only, sign in, or make a first purchase or stream), what is it worth, and where can it be spent? - Is the offer limited to members who have never installed the app, and in which countries and on which phone platforms does it run? - What does more app adoption do for the business: more viewing or purchases, stronger Prime renewal, or something else? The answer decides the success metric. ### What a Strong Answer Covers - A testable hypothesis and a primary metric that captures incremental, lasting app use rather than raw downloads - The eligible population, the randomization unit and a no-offer control, analyzed by assignment rather than by who chose to redeem - Sample size, a test length long enough to see behavior after the gift card is spent, and decision criteria fixed before launch - Guardrails and economics: cost per incremental retained user, uninstalls, activity that merely moves from other devices to the app, and abuse of the offer - Measurement pitfalls such as install attribution across devices and shared accounts, novelty effects and differences between member segments ### Follow-up Questions - Installs in the treatment group rise sharply, but app use a month later matches the control. What do you conclude and recommend? - How would you choose the gift card's value, and would you test more than one value? - Marketing wants every eligible member to get the offer, so there can be no holdout. How would you estimate the promotion's effect?