How to evaluate adding video ads in a game
Overview
This question evaluates skills in product analytics, experimentation design, causal inference and monetization modeling for free-to-play mobile games, including metrics selection, A/B testing, and lifetime value estimation.
Amazon
Dec 9, 2025, 12:00 AM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0
Loading...
Case study: Evaluate adding video ads to a mobile game
You are the Data Scientist for a free-to-play mobile game. The product team wants to add video ads (could be rewarded and/or interstitial) to increase monetization.
Task
Describe how you would evaluate whether adding video ads is a good idea, and how you would decide whether to roll it out.
Your answer should cover:
-
Goal & decision
- What is the decision to be made (ship / not ship / ship with constraints)?
- What does “success” mean?
-
Metrics (with tradeoffs)
- Propose primary metric(s) and explain why.
- Propose diagnostic metric(s) to understand mechanisms.
- Propose guardrail metric(s) to ensure you don’t harm the game.
-
Experiment / identification strategy
- How would you design an A/B test (unit of randomization, duration, ramp, segmentation/stratification)?
- What confounders or biases are likely (e.g., seasonality, payer vs non-payer mix, novelty effects), and how would you address them?
-
Analysis plan
- How would you estimate the incremental impact on total revenue and long-term value (LTV)?
- How would you handle heavy-tailed monetization metrics and delayed impact?
-
Rollout & monitoring
- What rollout plan would you recommend (e.g., frequency caps, targeting rules)?
- What would you monitor post-launch?
State any assumptions you need to make.
Solution
Login required
Show