End-to-End ML System Design: Real-Time Ad Selection
Context
You need to design a real-time, data-driven ad selection system that personalizes ads for millions of users. Assume:
-
Request volume: millions of users/day; peak QPS in the thousands.
-
Inventory: tens to hundreds of thousands of ads.
-
Strict latency targets: p95 < 100 ms for ad decisioning (excluding network).
-
Business objective: maximize expected value (e.g., clicks or revenue) while respecting advertiser budgets and policies.
Task
Design an end-to-end ML system to select ads at request time. Discuss and justify:
-
Data collection and logging
-
Labeling and outcome definition
-
Feature engineering and a feature store
-
Model choice and training objectives
-
Exploration–exploitation strategy (e.g., contextual bandits)
-
Offline evaluation and counterfactual/replay evaluation
-
Online evaluation (A/B tests, guardrails)
-
Latency and reliability constraints
-
Scalability and architecture (training/serving pipelines)
Hints: CTR prediction with logistic regression or GBDT; counterfactual evaluation (IPS/DR); feature store for train-serve consistency; bandits for exploration.