Design of a Distributed Search

The initial design partitions documents across nodes and runs indexing and queries on those same nodes. Replication adds alternative serving copies. The next lesson will examine when separating index construction from query serving is worth its publication and transfer costs.

High-level design

A distributed search system operates in two phases. The offline phase handles data crawling and indexing in the background. The online phase serves user search queries.