Design experiment for ads in chat with budgets

Q: Design experiment for ads in chat with budgets

This question evaluates a data scientist's skills in experimental design, causal inference, metrics selection, and handling allocation constraints for ad monetization incrementality.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Loading...

Context

After launching an AI chat feature, the company wants to show ads inside the chat experience (you are interviewing for an Ads/Monetization data science role). Advertisers have fixed budgets, so shifting impressions to chat may reallocate spend from other placements.

Task

Design an experiment to measure the incremental impact of introducing chat ads.

Requirements

Define the main incrementality question (platform revenue vs advertiser outcomes vs user experience).
Propose an A/B (or alternative) design that addresses:
- budget constraints / budget reallocation (“budget split”)
- auction dynamics and supply shift
- user-level effects and advertiser-level effects
Specify primary, diagnostic, and guardrail metrics .
Describe how you would interpret results under different scenarios (e.g., revenue up but ROAS down).

Design experiment for ads in chat with budgets

Context

Task

Requirements

Solution

Comments (0)

Design experiment for ads in chat with budgets

Overview

Context

Task

Requirements

Solution

Comments (0)