This question evaluates competency in designing and evaluating ranking models for personalized marketing emails, experiment design and online A/B testing, and statistical analysis of user engagement KPIs within the Machine Learning domain. It is commonly asked to assess end-to-end product ML thinking—balancing offline evaluation metrics (e.g.
Designing, deploying, and evaluating ranking models and marketing emails for Prime Video.
How would you approach sending marketing emails to customers to introduce a new Prime Video series? Outline your first steps, data requirements, and modeling approach. If the ranking function changes, how would you test whether the new function performs better? When you have several ranking functions, how would you determine which one is best?
Discuss experiment design, offline evaluation metrics (e.g., NDCG, MAP), online A/B testing, statistical significance, and user engagement KPIs.
Login required