Deep dive: LLM text classifier distilled through two levels of teachers, with metrics
Company: Yahoo
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: easy
Interview Round: Onsite
In a project deep-dive round, you present a project you worked on: a text classification system built with a large language model, where the model that ships was obtained by knowledge distillation through two levels of teachers. The interviewer has not seen the project before, asks you to walk through it, and then probes the detailed metrics. Prepare this deep dive so that a listener can follow the two-level distillation on first hearing, the design choices are justified, and every number you quote can be defended.
The report says only that the distillation had two levels of teachers. This prompt takes the usual reading, a cascade in which a large model teaches an intermediate model that in turn teaches the deployed student; adapt it if your own project is structured differently.
### Clarifying Questions
- How long is the deep dive, and should I draw the pipeline while I talk?
- Is the interviewer more interested in the modeling, the production system, or my individual contribution?
- Which metrics matter most to this team: offline quality, online impact, or serving cost and latency?
### Part 1 — Frame the project
Open with the problem, the constraints and the outcome in about two minutes, before any distillation detail.
```hint Headline first
Think about the one sentence that says what was classified, why the large model alone could not ship, and what the final model achieved.
```
#### What This Part Should Cover
- The product problem and the classification task: labels, inputs and volume
- The constraint that motivated distillation, such as latency, cost or throughput
- The final outcome in numbers, and your own role in it
### Part 2 — Explain the two-level distillation
Describe each model in the chain, what it is trained on, what it produces for the next stage, and why an intermediate teacher was used instead of distilling the large model straight into the student.
```hint One line per stage
For each stage, say only its input, its training signal and its output; keep loss weights and hyperparameters for when the interviewer asks.
```
#### What This Part Should Cover
- What each model is, and the labels or soft targets passed from one stage to the next
- The reason for the intermediate teacher, and the evidence that it helped
- How label noise and the large model's errors are kept from propagating
### Part 3 — Defend the detailed metrics
The interviewer asks for the detailed metrics. Present the metrics you used at each stage and online, how they were measured, and how they tie back to the product goal.
```hint Every number needs a denominator
For each metric you quote, be ready to say which data it was computed on, how large that set was, how its labels were made, and what it was compared with.
```
#### What This Part Should Cover
- Quality metrics for each stage on a held-out labeled set, including per-class and rare-class behavior
- How closely each student matches its teacher, and how much quality the deployed model retains relative to the large model
- Latency, throughput and cost at serving time, and the online result
### What a Strong Answer Covers
- A top-down narrative, with a simple diagram, that a listener new to the project can follow
- A clear separation between the candidate's own contributions and the team's
- Design choices justified by the alternatives considered and by experimental evidence
- An honest account of what was hard or failed and what would be done differently
### Follow-up Questions
- Why not distill the large model directly into the student? What experiment would show the intermediate teacher was worth its cost?
- How do you know the student is not inheriting systematic mistakes from the large model's labels?
- How did you choose the decision threshold, and how do the metrics behave for the rarest classes?
- If you had ten minutes to present this to a non-ML audience, what would you leave out?
Overview: Prepare a project deep dive on a text classifier built with a large language model and distilled through two levels of teachers into a small production model. Tests how clearly you frame the project, justify the intermediate teacher, and defend per-stage quality, latency, cost and online metrics.
Read the full Yahoo Machine Learning Engineer interview experience this question came from