Indexing in a Distributed Search
In this lesson3 sections
Indexing in a Distributed Search
An inverted index changes the question from “Which documents should I scan?” to “Which documents contain these terms?” Work through a small index first, then consider what happens when its storage and query load outgrow one server.
This lesson explores the mechanics of indexing and the transition from centralized to distributed architectures.
Indexing
Indexing organizes data to facilitate fast and accurate information retrieval.
Build a searchable index
A basic approach assigns a unique ID to each document and stores the text in a database table (a forward index). The first column holds the document ID, and the second contains the text.
Example source documents
| ID | Document Content |
|---|---|
| 1 | Elasticsearch is the distributed and analytics engine that is based on REST APIs. |
| 2 | Elasticsearch is a Lucene library-based search engine. |
| 3 | Elasticsearch is a distributed search and analytics engine built on Apache Lucene. |
This table maps document IDs to content. Without a supporting full-text index, finding a term requires inspecting document text. The problem is the access path, not the fact that a database table stores the text; databases can also maintain full-text indexes.
The rest of this lesson is for Premium members
Premium opens every lesson in this course, along with all questions and solutions on PracHub.