Explain Multi-Armed Bandit Principles

Quick Overview

This question evaluates understanding of multi-armed bandit principles and contextual bandits, covering algorithmic trade-offs (regret, exploration–exploitation balance, and modeling assumptions) among epsilon-greedy, UCB, and Thompson sampling, along with operational concerns such as delayed or batched rewards, non‑stationarity, offline policy evaluation, and production safety. It is commonly asked in Analytics & Experimentation and machine learning interviews because it probes both conceptual understanding and practical application of online decision-making, testing the ability to reason about algorithm selection, performance trade-offs, and deployment considerations.

Explain Multi-Armed Bandit Principles

Company: Amazon

Role: Machine Learning Engineer

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

Explain multi-armed bandit algorithms and when to use them over A/B tests. Compare epsilon-greedy, UCB, and Thompson sampling in terms of regret, exploration–exploitation balance, and assumptions. Extend to contextual bandits, discuss delayed or batched rewards, non-stationarity and drift handling, offline policy evaluation, and safety/guardrails for production deployment.

Quick Answer: This question evaluates understanding of multi-armed bandit principles and contextual bandits, covering algorithmic trade-offs (regret, exploration–exploitation balance, and modeling assumptions) among epsilon-greedy, UCB, and Thompson sampling, along with operational concerns such as delayed or batched rewards, non‑stationarity, offline policy evaluation, and production safety. It is commonly asked in Analytics & Experimentation and machine learning interviews because it probes both conceptual understanding and practical application of online decision-making, testing the ability to reason about algorithm selection, performance trade-offs, and deployment considerations.

|Home/Analytics & Experimentation/Amazon
Amazon logo
Amazon
Sep 6, 2025, 12:00 AM
hardMachine Learning EngineerOnsiteAnalytics & Experimentation
6
0

Multi-Armed Bandits vs A/B Testing: Algorithms, Trade-offs, and Production Considerations

You are designing online decision-making for a large-scale product (e.g., recommendations, pricing, notifications) where you must learn from user interactions while maximizing outcomes.

  1. Explain what multi-armed bandit (MAB) algorithms are and when to use them instead of standard A/B tests.
  2. Compare the following algorithms along three dimensions: regret, exploration–exploitation balance, and assumptions.
    • Epsilon-greedy
    • Upper Confidence Bound (UCB)
    • Thompson sampling (TS)
  3. Extend the discussion to contextual bandits: what they are, typical algorithms, and when to use them.
  4. Discuss operational considerations:
    • Delayed or batched rewards
    • Non-stationarity and drift handling
    • Offline policy evaluation (OPE) using logged bandit data
    • Safety and guardrails for production deployment
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Amazon•More Machine Learning Engineer•Amazon Machine Learning Engineer•Amazon Analytics & Experimentation•Machine Learning Engineer Analytics & Experimentation