Compute precision/recall from a flaky top-k API
Company: Microsoft
Role: Machine Learning Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Technical Screen
You have 10 image files. Each file has a ground-truth label indicating whether it contains a dog.
You can call an API like `searchDogs(k)` which is intended to return **k file IDs** that the system predicts are dogs (e.g., top-k results for the query "dog").
Tasks:
1) Write pseudocode to compute **precision** and **recall** of the API’s returned results with respect to the ground truth.
2) Follow-up: the API may behave unexpectedly (returns `None`, throws, returns fewer than k items, returns more than k items, returns duplicates, or returns unknown file IDs). How would you handle these cases so the metric computation is robust and well-defined?
Overview: This question evaluates understanding of binary classification evaluation metrics (precision and recall) and competency in handling unreliable API behavior for robust metric computation, testing both conceptual understanding and practical application.
Read the full Microsoft Machine Learning Engineer interview experience this question came from