Choose Fast or Cheap Models

You are building an AI-powered product and must choose between two inference options for each request: - **Option A:** higher cost per token, but lower latency - **Option B:** lower cost per token, but higher latency How would you decide when to use each option? Discuss the trade-offs across user experience, latency, quality, reliability, and operating cost. Also explain what metrics you would track, how you would segment different workloads, and whether you would use a dynamic routing strategy instead of a single global choice.