Design an ML Forecasting and Pacing System for Ad Bidding
Company: LinkedIn
Role: Machine Learning Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Technical Screen
## Prompt
Design an ad-bidding pacing system that uses forecasts to allocate campaign budget over time while still enforcing hard spend and policy constraints with deterministic rules. Cover labels, features, offline training, online inference, the pacing controller, delayed feedback, drift, and safe fallback behavior.
### Constraints & Assumptions
- Campaigns have budgets, start/end times, targeting constraints, and a delivery objective.
- The rule-based controller owns hard budget caps; ML supplies forecasts or calibrated response estimates.
- Auction outcomes and conversions can arrive late and may be censored.
- A model outage must degrade to a safe deterministic pacing policy.
### Clarifying Questions to Ask
- Is the primary objective smooth budget delivery, clicks, conversions, value, or a constrained combination?
- At what cadence can bids or pacing multipliers change without destabilizing delivery?
- Which feedback is available immediately at bid time and which labels are delayed?
```hint Separate prediction from control
A forecast estimates what will happen under actions; a pacing controller chooses an action subject to remaining budget, time, and risk bounds.
```
```hint Backtest through time
Random row splits leak future auction conditions. Train and evaluate on chronological windows with point-in-time feature joins.
```
### What a Strong Answer Covers
- Problem formulation and decomposition into traffic, win-rate, cost, and outcome forecasts.
- Point-in-time-correct features, labels, delayed-feedback handling, and offline evaluation.
- Online prediction path and a controller that converts forecasts into bounded actions.
- Calibration, uncertainty, exploration, drift, retraining, and guardrails.
- Shadow testing, online experiments, fallback, monitoring, and rollback.
### Follow-up Questions
1. How would you avoid oscillation when every campaign reacts to the same traffic forecast?
2. How would cold-start campaigns receive usable forecasts?
3. Which online metric would trigger automatic fallback to rule-based pacing?
Overview: Design an ML forecasting and pacing system for ad bidding that keeps hard budget controls deterministic while handling time-correct training, delayed feedback, calibrated actions, drift, experiments, and safe fallback.