Measure and Improve Listing Quality with Key Metrics
Overview
This question evaluates competency in product analytics, metric definition, experimentation, and the operationalization of measurement systems to quantify and improve listing quality in an online marketplace.
Etsy
Aug 4, 2025, 10:55 AM
Data Scientist
Technical Screen
Analytics & Experimentation
43
0
Evaluating and Improving Listing Quality in an Online Marketplace
Scenario
You are a product/analytics lead at an online marketplace (e.g., Airbnb, Etsy) responsible for evaluating and improving the quality of individual listings shown to users across search, recommendations, and ads.
Task
Explain why measuring listing quality matters to the business. Define specific metrics to quantify listing quality (e.g., user engagement, conversion rate, content completeness, complaints). Then outline how you would operationalize these metrics and set up a continuous measurement and improvement program.
Questions
- Why is it important for the business to measure listing quality?
- Which specific metrics would you define to quantify listing quality?
- How would you operationalize these metrics and establish a continuous measurement and improvement approach?
Hints
- Tie outcomes to user experience and revenue.
- Consider engagement, conversion funnels, search ranking, content health, trust/safety, and controlled experiments.
- Address position bias, category/price differences, cold-start, and low-sample listings.
Solution
Login required
Show