Pokémon Go PvP Battles: Pick a North-Star Metric, Funnel Metrics and Guardrails
Company: Niantic
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
Pokémon Go lets players battle each other with the Pokémon they have collected, a player-versus-player (PvP) feature. You are asked how you would measure whether this feature is a success. Define what success means for PvP, choose the main metric you would use, and describe the supporting metrics and guardrails you would track alongside it.
```hint Start from the feature's job
Ask what a battle mode adds to a game built around walking, catching and collecting, then pick a metric that rises only when that purpose is being met.
```
### Clarifying Questions
- Does PvP here mean casual battles between friends, a ranked online league, or both?
- Is the feature newly launched or long established? That decides whether adoption or retention matters most.
- Should success be judged only within PvP, or also by its effect on the rest of the game, such as walking, catching and raids?
- Is the main business aim engagement and retention, or revenue from in-game purchases?
### What a Strong Answer Covers
- A stated purpose for PvP within the game, linked to a single primary metric.
- A funnel from first exposure to a first battle to repeat play, with a metric at each step.
- Effects on the rest of the game, including whether PvP replaces exploring and catching.
- Guardrails on match fairness, connection quality, cheating and player frustration.
- How to separate the feature's effect from seasons, events and the self-selection of engaged players.
### Follow-up Questions
- Battles per player rose sharply after a rewards change, but 30-day retention of battlers fell. How do you read that?
- How would you run an experiment on a PvP change when players in the test and control groups battle each other?
- PvP activity is concentrated in a small group of highly competitive players. Is that a success or a problem?
Overview: A product metrics prompt asking how to measure whether the player-versus-player battle feature in Pokémon Go is succeeding. It tests defining the feature's purpose, choosing a primary metric with funnel and guardrail metrics, and separating the feature's real impact from seasonal events and player self-selection.