Self-Hosted Open-Weight LLM vs. Paid Model API: Making the Build-or-Buy Call

Your team needs a large language model to power a product feature. You can run an open-source model on your own infrastructure, or you can call a commercial API such as OpenAI's models or Anthropic's Claude. How would you choose? ```hint Separate gates from weights Some factors can rule an option out on their own, such as where user data is allowed to go. Others only tilt the balance, such as price per token. Sort the factors for this feature into those two groups before comparing anything. ``` ### Clarifying Questions - What is the feature, and does it serve Snapchat-scale consumer traffic, such as an in-app assistant, or is it a lower-volume internal tool? - What data will the model see: private messages and photos, or only public or internal content? - How much model-serving and fine-tuning experience does the team already have? - Is the priority shipping quickly, or long-term unit cost and control over the model? ### What a Strong Answer Covers - Decision criteria tied to this feature: quality on the actual task, cost at expected volume, latency, data privacy and compliance, control over model behavior, and team capacity. - A cost comparison by volume that counts fixed engineering and idle-capacity costs, not just per-token prices. - An evaluation plan that tests candidate models on the product's own tasks and safety cases before committing. - Risks on each path, such as vendor lock-in, price or model changes, and the burden of running and securing your own stack, and how the architecture keeps the decision reversible. ### Follow-up Questions - You launched on an API, and six months later usage is ten times the forecast. What do you check before moving to self-hosting? - Legal rules that user messages may not leave company infrastructure. How does the decision change? - How would you run both options at once, and how would you route requests between them?