Choosing Which Model Capabilities to Improve Next in Claude Sonnet
Company: Anthropic
Role: Product Manager
Category: Product Design & Strategy
Difficulty: medium
Interview Round: Other
You are a product manager at Anthropic responsible for Claude Sonnet, the middle tier of the Claude model family, between the smaller Haiku and the larger Opus. How would you improve Sonnet as a foundation model?
```hint Improve the model, not the interface
Pick the customers whose results depend most on how Sonnet itself behaves, find where the model lets them down in real use, and favor improvements you could prove with an evaluation before release.
```
### Clarifying Questions
- Should the answer stay with the model's own behavior (capabilities, speed, cost, reliability, safety), or may it include the API features and apps built around it?
- Which customers matter most this round: developers building on the API, enterprises deploying through cloud partners, or people using Claude's own apps?
- Is the main goal more usage and revenue, keeping existing customers through model upgrades, or closing a gap with competing models?
- How should the work relate to the other tiers: is it acceptable for Sonnet to improve in ways that pull some customers away from Opus?
### What a Strong Answer Covers
- A goal tied to Sonnet's role as the tier that balances quality, speed and price, and a clearly chosen customer segment
- Pain points grounded in how that segment uses a model in production, such as long multi-step tasks, tool calls, cost at volume and behavior changes between versions
- Concrete model-level improvements, prioritized with reasons, and how each would be evaluated before release
- Success metrics spanning model evaluations, customer outcomes and business results, with safety and over-refusal guardrails
- Risks and trade-offs, including misuse of stronger capabilities, benchmark overfitting, compute cost and positioning against the other tiers
### Follow-up Questions
- Your top improvement raises success on agentic tasks but makes each request slower and more expensive. How do you decide whether it ships in Sonnet or waits for Opus?
- An enterprise customer says the last Sonnet upgrade broke their prompts. What would you change in how new versions are released?
- How would you know whether an improvement that looks good on internal evaluations actually helps customers?
Overview: A product sense prompt asking how Anthropic should improve Claude Sonnet at the level of the model itself. It tests choosing a customer segment, finding where a mid-tier foundation model falls short in production use, prioritizing improvements, and defining evaluations, success metrics and safety guardrails.