Describing a data-driven change to ad serving: analysis, decision, test and outcome

You are interviewing for a product manager role at Rakuten and are asked to describe a situation in which data led to a new or changed ad serving decision, such as who saw an ad, which ad was chosen, where or when it appeared, or how often it was shown. Walk through what the data showed, how it was analyzed, what changed in the way ads were served, how the change was checked, and what resulted. ```hint Name the lever Be precise about what was served differently afterward: a targeting rule, a ranking weight, a frequency cap, the number or position of ad slots, or a pacing setting. Tie that lever to the specific pattern in the data that justified moving it. ``` ### Clarifying Questions - Should the situation come from your own work, or is a realistic hypothetical acceptable if you have not worked on ad serving directly? - Is the interest in the advertiser side (campaign targeting and performance) or the publisher side (ad load, placement and the experience of people who see the ads)? - Does "ad serving decision" include ranking, auction and pacing settings, or only targeting and placement? ### What a Strong Answer Covers - A specific situation with its business context, the metric that signaled a problem or opportunity, and the candidate's own role. - The analysis: which data was used, how the issue was localized by segment, and how competing explanations were ruled out. - The exact serving lever that changed, why it was chosen over alternatives, and how stakeholders such as advertisers were handled. - Validation through an experiment or staged rollout, with a primary metric and guardrails for viewers, advertisers and revenue. - A quantified result, the decision that followed, and what the candidate learned. ### Follow-up Questions - What result would have convinced you the change was a mistake, and how quickly could you have rolled it back? - If the change improved the experience for people seeing the ads but cut short-term ad revenue, how would you make the case for keeping it? - How did you make sure the result was not driven by seasonality or by a handful of large advertisers?