Improve an ML Inference Service
Company: Anthropic
Role: Software Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Onsite
Overview: Improve an ML inference service with cost-aware admission, dynamic batching, load shedding, scoped caching, and exact idempotency semantics.