Deep dive: LLM text classifier distilled through two levels of teachers, with metrics

Quick Overview

Prepare a project deep dive on a text classifier built with a large language model and distilled through two levels of teachers into a small production model. Tests how clearly you frame the project, justify the intermediate teacher, and defend per-stage quality, latency, cost and online metrics.

Deep dive: LLM text classifier distilled through two levels of teachers, with metrics

Company: Yahoo

Role: Machine Learning Engineer

Category: Machine Learning

Difficulty: easy

Interview Round: Onsite

In a project deep-dive round, you present a project you worked on: a text classification system built with a large language model, where the model that ships was obtained by knowledge distillation through two levels of teachers. The interviewer has not seen the project before, asks you to walk through it, and then probes the detailed metrics. Prepare this deep dive so that a listener can follow the two-level distillation on first hearing, the design choices are justified, and every number you quote can be defended. The report says only that the distillation had two levels of teachers. This prompt takes the usual reading, a cascade in which a large model teaches an intermediate model that in turn teaches the deployed student; adapt it if your own project is structured differently. ### Clarifying Questions - How long is the deep dive, and should I draw the pipeline while I talk? - Is the interviewer more interested in the modeling, the production system, or my individual contribution? - Which metrics matter most to this team: offline quality, online impact, or serving cost and latency? ### Part 1 — Frame the project Open with the problem, the constraints and the outcome in about two minutes, before any distillation detail. ```hint Headline first Think about the one sentence that says what was classified, why the large model alone could not ship, and what the final model achieved. ``` #### What This Part Should Cover - The product problem and the classification task: labels, inputs and volume - The constraint that motivated distillation, such as latency, cost or throughput - The final outcome in numbers, and your own role in it ### Part 2 — Explain the two-level distillation Describe each model in the chain, what it is trained on, what it produces for the next stage, and why an intermediate teacher was used instead of distilling the large model straight into the student. ```hint One line per stage For each stage, say only its input, its training signal and its output; keep loss weights and hyperparameters for when the interviewer asks. ``` #### What This Part Should Cover - What each model is, and the labels or soft targets passed from one stage to the next - The reason for the intermediate teacher, and the evidence that it helped - How label noise and the large model's errors are kept from propagating ### Part 3 — Defend the detailed metrics The interviewer asks for the detailed metrics. Present the metrics you used at each stage and online, how they were measured, and how they tie back to the product goal. ```hint Every number needs a denominator For each metric you quote, be ready to say which data it was computed on, how large that set was, how its labels were made, and what it was compared with. ``` #### What This Part Should Cover - Quality metrics for each stage on a held-out labeled set, including per-class and rare-class behavior - How closely each student matches its teacher, and how much quality the deployed model retains relative to the large model - Latency, throughput and cost at serving time, and the online result ### What a Strong Answer Covers - A top-down narrative, with a simple diagram, that a listener new to the project can follow - A clear separation between the candidate's own contributions and the team's - Design choices justified by the alternatives considered and by experimental evidence - An honest account of what was hard or failed and what would be done differently ### Follow-up Questions - Why not distill the large model directly into the student? What experiment would show the intermediate teacher was worth its cost? - How do you know the student is not inheriting systematic mistakes from the large model's labels? - How did you choose the decision threshold, and how do the metrics behave for the rarest classes? - If you had ten minutes to present this to a non-ML audience, what would you leave out?

Overview: Prepare a project deep dive on a text classifier built with a large language model and distilled through two levels of teachers into a small production model. Tests how clearly you frame the project, justify the intermediate teacher, and defend per-stage quality, latency, cost and online metrics.

Read the full Yahoo Machine Learning Engineer interview experience this question came from

Yahoo logo
Yahoo
Nov 8, 2026
easyMachine Learning EngineerOnsiteMachine Learning
0
0

In a project deep-dive round, you present a project you worked on: a text classification system built with a large language model, where the model that ships was obtained by knowledge distillation through two levels of teachers. The interviewer has not seen the project before, asks you to walk through it, and then probes the detailed metrics. Prepare this deep dive so that a listener can follow the two-level distillation on first hearing, the design choices are justified, and every number you quote can be defended.

The report says only that the distillation had two levels of teachers. This prompt takes the usual reading, a cascade in which a large model teaches an intermediate model that in turn teaches the deployed student; adapt it if your own project is structured differently.

Clarifying Questions Guidance

  • How long is the deep dive, and should I draw the pipeline while I talk?
  • Is the interviewer more interested in the modeling, the production system, or my individual contribution?
  • Which metrics matter most to this team: offline quality, online impact, or serving cost and latency?

Part 1 — Frame the project

Open with the problem, the constraints and the outcome in about two minutes, before any distillation detail.

What This Part Should Cover Guidance

  • The product problem and the classification task: labels, inputs and volume
  • The constraint that motivated distillation, such as latency, cost or throughput
  • The final outcome in numbers, and your own role in it

Part 2 — Explain the two-level distillation

Describe each model in the chain, what it is trained on, what it produces for the next stage, and why an intermediate teacher was used instead of distilling the large model straight into the student.

What This Part Should Cover Guidance

  • What each model is, and the labels or soft targets passed from one stage to the next
  • The reason for the intermediate teacher, and the evidence that it helped
  • How label noise and the large model's errors are kept from propagating

Part 3 — Defend the detailed metrics

The interviewer asks for the detailed metrics. Present the metrics you used at each stage and online, how they were measured, and how they tie back to the product goal.

What This Part Should Cover Guidance

  • Quality metrics for each stage on a held-out labeled set, including per-class and rare-class behavior
  • How closely each student matches its teacher, and how much quality the deployed model retains relative to the large model
  • Latency, throughput and cost at serving time, and the online result

What a Strong Answer Covers Guidance

  • A top-down narrative, with a simple diagram, that a listener new to the project can follow
  • A clear separation between the candidate's own contributions and the team's
  • Design choices justified by the alternatives considered and by experimental evidence
  • An honest account of what was hard or failed and what would be done differently

Follow-up Questions Guidance

  • Why not distill the large model directly into the student? What experiment would show the intermediate teacher was worth its cost?
  • How do you know the student is not inheriting systematic mistakes from the large model's labels?
  • How did you choose the decision threshold, and how do the metrics behave for the rarest classes?
  • If you had ten minutes to present this to a non-ML audience, what would you leave out?