Review and Scale an Inference Service Design
Company: Anthropic
Role: Software Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Technical Screen
Overview: Review a high-throughput inference design with a 100 ms batch target, explicit capacity math, overload control, and multi-model scheduling.