Trading model quality for latency or cost and explaining the call to leadership
Company: Yahoo
Role: Machine Learning Engineer
Category: Behavioral & Leadership
Difficulty: easy
Interview Round: Onsite
In a behavioral round with a product manager, a senior machine learning engineer is asked: "Tell me about a time you traded model quality for latency or cost, or another functional versus non-functional metric." The follow-up: "How did you explain the trade-off to product or leadership, and who made the call?"
### Clarifying Questions
- Should the story be one where you made the decision, or one where you shaped a decision someone else made?
- How much technical detail about the optimizations is useful for this audience?
### Part 1 — The trade-off
"Tell me about a time you traded model quality for latency or cost, or another functional versus non-functional metric."
```hint Show the curve
Think about whether you compared several points on the quality-versus-latency or quality-versus-cost curve, or only two options.
```
#### What This Part Should Cover
- The constraint, and why it mattered to users or the business
- The options considered, quantified on both axes
- The chosen point, and what was given up
### Part 2 — Explaining it and making the call
"How did you explain the trade-off to product or leadership, and who made the call?"
```hint Speak in the listener's units
Ask what a quality drop of a few points means in user-visible terms, and what the latency or cost saving buys.
```
#### What This Part Should Cover
- How the trade-off was presented to non-ML decision makers
- Who decided, and on what basis
- How the decision was checked after launch
### What a Strong Answer Covers
- A real decision with numbers on both sides of the trade-off
- Attempts to shrink the trade-off before accepting it
- Clear decision rights and communication
- Follow-through: monitoring the outcome and revisiting the choice
### Follow-up Questions
- What would have made you refuse the trade-off?
- Did you try to win the quality back later, and how?
- How did you make sure the quality loss did not fall mostly on one group of users?
- If leadership had chosen the option you argued against, what would you have done?
Overview: Behavioral question for a senior machine learning engineer: describe a time you traded model quality for latency, cost or another non-functional metric. Tests how the options were quantified, how the trade-off was explained to product or leadership, and who made the final call.
Read the full Yahoo Machine Learning Engineer interview experience this question came from