System Design: The Distributed Search

A search system does expensive preparation before a query arrives. This chapter follows content from collection to an inverted index, then shows how several servers cooperate to return relevant results.

Why do we need a search system?

A search bar lets users describe what they want instead of browsing every item. On a course site, a learner may know a topic but not its course title. Search connects that query to relevant content; categories and browsing remain useful when the learner does not yet know what to ask for.

Large video catalogs and web collections make exhaustive manual browsing impractical. A search engine finds and ranks candidates within a latency budget. Small collections may fit on one server; distribution becomes useful when index size, query load, or availability requirements exceed that setup.