Debug online worse than offline model performance

Q: Debug online worse than offline model performance

This is a ML System Design interview question from Amazon for Machine Learning Engineer roles. View the full question and solution on PracHub.

Q: How do I approach ML System Design interview questions?

ML System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master ml system design interviews.

Question

Loading...

Production ML: online performance worse than offline

You launch an ML model. Offline evaluation (validation/test) looked good, but after deployment the online metrics are significantly worse.

Question

What step-by-step troubleshooting order would you follow to identify and fix the issue? Include what you would check in data, features, serving, evaluation, and feedback loops.

Debug online worse than offline model performance

Production ML: online performance worse than offline

Question

Comments (0)