Design an ads ranking system with calibration

Q: Design an ads ranking system with calibration

This question evaluates a candidate's ability to design scalable, low-latency online machine learning systems for ads ranking, covering competencies in feature engineering, ranking model architecture, probability calibration, training and serving pipelines, and monitoring.

Q: How do I approach ML System Design interview questions?

ML System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master ml system design interviews.

Question

Loading...

ML System Design: Ads Ranking (e-commerce)

Design an online ads ranking (ad “re-ranking”) system for an e-commerce app.

The system receives a request when a user opens a page/feed and must select and order a set of candidate ads to show.

Requirements

Objective: maximize long-term business value (e.g., revenue), while maintaining user experience
Latency: low-latency online ranking (tens of milliseconds to a few hundred ms, depending on assumptions)
Scale: many users/requests, many advertisers/items
Modeling topics to cover:
- Feature engineering (user, item/ad, context, cross features)
- Model architecture choices for ranking
- Calibration of predicted probabilities (e.g., CTR/CVR) and why it matters
Evaluation: offline metrics + online A/B testing and guardrails

Explain your end-to-end design: candidate generation, ranking/re-ranking, training pipeline, serving, and monitoring.

Design an ads ranking system with calibration

Quick Overview

ML System Design: Ads Ranking (e-commerce)

Requirements

Solution

Comments (0)