Improve Keyword Search Ranking
Company: Box
Role: Machine Learning Engineer
Category: ML System Design
Difficulty: hard
Interview Round: Onsite
You are working on a search product that currently uses a keyword-based index, such as an inverted index with lexical matching. Users can usually find relevant documents, but the best result is often ranked far below the top positions, causing poor user satisfaction.
Design an improved search ranking system that keeps the existing keyword index as a candidate-generation layer but improves the ordering of results.
Address the following:
- How would you diagnose why relevant results are ranked too low?
- What ranking features would you add beyond keyword matching?
- How would you use machine learning, if appropriate, to improve relevance?
- How would you collect labels or feedback data?
- How would you serve the system with low latency?
- What offline and online metrics would you use to evaluate success?
- What follow-up issues would you consider, such as cold start, personalization, freshness, spam, and fairness?
Quick Answer: This question evaluates expertise in designing ML-driven search ranking systems, covering information retrieval, feature engineering, relevance modeling, feedback and label collection, offline and online evaluation, low-latency serving, and operational concerns like cold start, personalization, freshness, spam, and fairness.