Search Words and Phrases in a Large Line-Oriented File
Quick Overview
Design word and exact-phrase search over a line-oriented file that may be much larger than memory. Compare a streaming one-off scan with a persistent positional inverted index, defining tokenization, Unicode and punctuation behavior, file-version binding, updates, result locations, and resource trade-offs.
Search Words and Phrases in a Large Line-Oriented File
Company: Confluent
Role: Software Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Onsite
# Search Words and Phrases in a Large Line-Oriented File
A text file contains one sentence per line and may be much larger than memory. Design an API that first supports finding every line containing a word and then supports finding every line containing an exact multiword phrase.
Discuss both a one-off search and a system that serves repeated queries against a mostly stable file. Define tokenization, phrase boundaries, case handling, result format, and how updates become searchable.
### Constraints & Assumptions
- Sequential reads are allowed; loading the complete file into memory is not.
- Results identify at least the line number and may include byte offsets.
- Exact behavior for punctuation and Unicode must be stated.
- The repeated-query design may use persistent auxiliary storage.
### Clarifying Questions to Ask
- Is a word matched as a token or as an arbitrary substring?
- Must phrase terms be adjacent and in the same line?
- Are queries rare enough for a full scan, or is low query latency required?
- Can the source file be appended to or rewritten while indexing?
### What a Strong Answer Covers
- A correct streaming scan for one-off queries
- Token and phrase semantics that remain consistent between indexing and lookup
- An inverted index with positions for repeated phrase queries
- Memory, CPU, storage, and update complexity
- Safe file-version binding, malformed input handling, and tests
### Follow-up Questions
- How would you return matches in relevance order rather than file order?
- How would you shard an index that no longer fits on one machine?
- What changes for compressed input?
- How can a new index version become visible atomically?
Quick Answer: Design word and exact-phrase search over a line-oriented file that may be much larger than memory. Compare a streaming one-off scan with a persistent positional inverted index, defining tokenization, Unicode and punctuation behavior, file-version binding, updates, result locations, and resource trade-offs.
Search Words and Phrases in a Large Line-Oriented File
A text file contains one sentence per line and may be much larger than memory. Design an API that first supports finding every line containing a word and then supports finding every line containing an exact multiword phrase.
Discuss both a one-off search and a system that serves repeated queries against a mostly stable file. Define tokenization, phrase boundaries, case handling, result format, and how updates become searchable.
Constraints & Assumptions
Sequential reads are allowed; loading the complete file into memory is not.
Results identify at least the line number and may include byte offsets.
Exact behavior for punctuation and Unicode must be stated.
The repeated-query design may use persistent auxiliary storage.
Clarifying Questions to Ask Guidance
Is a word matched as a token or as an arbitrary substring?
Must phrase terms be adjacent and in the same line?
Are queries rare enough for a full scan, or is low query latency required?
Can the source file be appended to or rewritten while indexing?
What a Strong Answer Covers Guidance
A correct streaming scan for one-off queries
Token and phrase semantics that remain consistent between indexing and lookup
An inverted index with positions for repeated phrase queries
Memory, CPU, storage, and update complexity
Safe file-version binding, malformed input handling, and tests
Follow-up Questions Guidance
How would you return matches in relevance order rather than file order?
How would you shard an index that no longer fits on one machine?
What changes for compressed input?
How can a new index version become visible atomically?