Design an AI-Powered Search System, Focusing on the Search Index

Design an AI-powered search system: users search in natural language and should find relevant content even when their words differ from the wording of that content. Most of the interview is about one component, the search index. Explain what you index, how the index is structured and partitioned, how a query uses it, and how it stays current as content changes. The interviewer keeps the discussion on the index and declines to spend time on other parts of the system, so cover the rest only as far as the index depends on it. ```hint What is one entry in the index? Decide what unit you embed and index: a whole item, a section, or a fixed-size chunk. Each choice changes recall, index size, and how a hit maps back to what the user sees. ``` ```hint Meaning is not everything Think about queries that embedding similarity handles badly, such as exact identifiers, names or quoted phrases, and what that implies for the index. ``` ```hint Results the user may not open Search must never surface content the user cannot access. Decide where that rule is enforced, and what has to change in the index when sharing settings change. ``` ### Constraints and Clarifications - The content being searched, the scale and the latency targets are not given; ask for them or state your assumptions. - Other components, such as the user interface, query understanding and answer generation, are out of scope except where they shape the index. ### Clarifying Questions - What content is searched (documents, tasks, comments, attachments), and is it split across many customer organizations? - How many items are indexed, how many queries arrive at peak, and what query latency is expected? - How quickly must an edit, a deletion or a permission change show up in search results? - Should the system only return ranked results, or also generate an answer from the retrieved content? - Which languages must the search support? ### What a Strong Answer Covers - A deliberate choice of indexing unit and record schema, including the metadata needed for filtering and ranking - A hybrid design that combines a keyword inverted index with vector embeddings, and a defined way to merge their results - An approximate nearest-neighbor structure, with its recall, memory and latency trade-offs - Partitioning and sharding of the index, including isolation between customers of very different sizes - Permission enforcement inside retrieval, and its behavior when access changes - Incremental updates and deletes, plus re-embedding the corpus without downtime when the embedding model changes - How the quality and health of the index are measured ### Follow-up Questions - The embedding model is replaced by a better one. How do you re-embed the whole corpus while search keeps working? - One customer has a thousand times more content than the median customer. How does that change your partitioning? - A user loses access to a document. How quickly does it disappear from their results, and how do you guarantee it? - How would you tell whether a change to chunking or to the index structure improved search quality?