How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a hard difficulty Machine Learning question, commonly asked during Technical Screen rounds at Upstart.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Upstart during technical interviews.

Design a Real-Time Personalized Ad Selection System

Quick Overview

This question evaluates competency in end-to-end real-time ML system design for personalized ad selection, including model decisioning, feature engineering and stores, exploration–exploitation strategies, offline and online evaluation, and scalability and latency constraints within the Machine Learning domain.

End-to-End ML System Design: Real-Time Ad Selection

Context

You need to design a real-time, data-driven ad selection system that personalizes ads for millions of users. Assume:

Request volume: millions of users/day; peak QPS in the thousands.
Inventory: tens to hundreds of thousands of ads.
Strict latency targets: p95 < 100 ms for ad decisioning (excluding network).
Business objective: maximize expected value (e.g., clicks or revenue) while respecting advertiser budgets and policies.

Task

Design an end-to-end ML system to select ads at request time. Discuss and justify:

Data collection and logging
Labeling and outcome definition
Feature engineering and a feature store
Model choice and training objectives
Exploration–exploitation strategy (e.g., contextual bandits)
Offline evaluation and counterfactual/replay evaluation
Online evaluation (A/B tests, guardrails)
Latency and reliability constraints
Scalability and architecture (training/serving pipelines)

Hints: CTR prediction with logistic regression or GBDT; counterfactual evaluation (IPS/DR); feature store for train-serve consistency; bandits for exploration.

Quick Overview

Context

You need to design a real-time, data-driven ad selection system that personalizes ads for millions of users. Assume:

Request volume: millions of users/day; peak QPS in the thousands.

Inventory: tens to hundreds of thousands of ads.

Strict latency targets: p95 < 100 ms for ad decisioning (excluding network).

Business objective: maximize expected value (e.g., clicks or revenue) while respecting advertiser budgets and policies.

Task

Design an end-to-end ML system to select ads at request time. Discuss and justify:

Data collection and logging

Labeling and outcome definition

Feature engineering and a feature store

Model choice and training objectives

Exploration–exploitation strategy (e.g., contextual bandits)

Offline evaluation and counterfactual/replay evaluation

Online evaluation (A/B tests, guardrails)

Latency and reliability constraints

Scalability and architecture (training/serving pipelines)

Hints: CTR prediction with logistic regression or GBDT; counterfactual evaluation (IPS/DR); feature store for train-serve consistency; bandits for exploration.

Design a Real-Time Personalized Ad Selection System

Quick Overview

End-to-End ML System Design: Real-Time Ad Selection

Context

Task

Solution

Comments (0)

Design a Real-Time Personalized Ad Selection System

Quick Overview

End-to-End ML System Design: Real-Time Ad Selection

Context

Task

Solution

Comments (0)