Design a Production ML Serving System

You are given an existing ML-powered production system that serves online user requests. The interview focuses **not** on changing the model architecture itself, but on how to operate the system reliably at scale. Design how you would **scale, monitor, and optimize** this system in production. Your discussion should cover: - The high-level serving architecture for online inference - How to scale the system as traffic grows - Reliability and fault tolerance strategies - Observability: what to log, monitor, and alert on - Performance optimization for latency, throughput, and cost - Safe rollout, evaluation, and rollback of model or infrastructure changes - How to detect and respond to production issues such as degraded quality, data drift, feature pipeline failures, or rising tail latency Assume the model is already trained and deployed, and the main goal is to run the ML system efficiently and safely in a real production environment.