Route and Batch Inference with Eight GPUs
Company: Anthropic
Role: Software Engineer
Category: ML System Design
Difficulty: hard
Interview Round: Technical Screen
Overview: Evaluate inference routing and batching, then schedule large and small models across eight GPUs with explicit resource limits and overload behavior.