Anthropic ML System Design Interview Questions
Master your tech interview with our curated database of real questions from top companies.
Design an LLM-based binary classifier
Design a Binary Text Classifier Using Only a Log-Probability Scoring Helper Context You are building a binary text classifier without fine-tuning. You...
Design an inference routing and scheduling layer
System Design: Routing Layer for Heterogeneous Inference Backends (GPU/CPU) Context You are asked to design a routing layer that sits between a user-f...
Design a low-latency ML inference API
System Design: Low‑Latency ML Inference API (Real‑Time) Context You are designing an in‑region, synchronous inference API used by product surfaces (e....
Design a batch inference API
System Design: Async Inference Service API (POST Job, Poll for Results) Context You are designing an asynchronous inference service where clients subm...
Review an inference API design for scale
System Design Review: Machine-Learning Inference API (Distributed Systems Focus) Background You are reviewing a teammate’s design document for a produ...
Design a GPU inference API
System Design: GPU-Backed Multi-Model Inference API Context Design a production-grade inference platform for serving multiple ML models (e.g., LLMs, v...
Design a prompt processing backend
System Design: Background Processing Backend for LLM Prompts Context Design a multi-tenant backend that processes large language model (LLM) prompts a...