Estimate unbiased ad scores with many reviewers
Company: Google
Role: Data Scientist
Category: Statistics & Math
Difficulty: medium
Interview Round: Technical Screen
Overview: Estimate each ad's latent quality, with uncertainty and a defensible ranking, from 10,000 ratings in which 100 reviewers each scored 100 of 1,000 ads on a 1-10 scale. Reviewers differ in harshness and in how widely they spread scores, and ads are rated by only partially overlapping reviewers, so comparability across ads is the core difficulty.
Read the full Google Data Scientist interview experience this question came from