Design an ML search system

Quick Overview

This question evaluates system-design and information-retrieval competencies for machine-learning-powered enterprise document search, covering ingestion and indexing pipelines, hybrid lexical and semantic retrieval, feature generation and ranking, multi-tenant access control, and security/compliance.

Design an ML search system

Company: OpenAI

Role: Machine Learning Engineer

Category: ML System Design

Difficulty: hard

Interview Round: Technical Screen

Design an ML-powered enterprise document search system. Specify requirements (query latency SLA, freshness, multi-tenant access control, data privacy/PII, cost). Describe the indexing pipeline (connectors, parsing/OCR, normalization), feature generation (BM25 signals, embeddings), storage (inverted index plus vector store) and a hybrid retrieval strategy. Detail the ranking stack (learning-to-rank or neural reranker), query understanding (spelling, synonyms, embeddings), and personalization/feedback loops. Explain online/offline evaluation (NDCG/Recall@k, interleaving/A/B), monitoring, and guardrails (sensitive-content filters). Address scalability, caching, result deduplication, multilingual support, and how you enforce ACLs at query time without leaking documents.

Quick Answer: This question evaluates system-design and information-retrieval competencies for machine-learning-powered enterprise document search, covering ingestion and indexing pipelines, hybrid lexical and semantic retrieval, feature generation and ranking, multi-tenant access control, and security/compliance.

|Home/ML System Design/OpenAI
OpenAI logo
OpenAI
Sep 6, 2025, 12:00 AM
hardMachine Learning EngineerTechnical ScreenML System Design
11
0

Design an ML‑Powered Enterprise Document Search System

Context

You are designing a multi‑tenant enterprise search system that indexes documents from many sources (e.g., Google Drive, SharePoint, Confluence, Slack, GitHub) and serves low‑latency, high‑relevance results to end users. The system must enforce strict access controls, protect sensitive data, and scale cost‑effectively.

Requirements

Specify and justify:

  1. Functional
    • Supported content types (PDF, Office docs, emails, images with text, code, web pages).
    • Query types (keyword, natural language questions, filters/facets, multilingual).
    • Multi‑tenant isolation and per‑document access control lists (ACLs).
    • Personalization (user/org context, recency, popularity).
  2. Non‑functional
    • Latency SLA (e.g., P50, P95, P99 for query and for reranking).
    • Freshness SLA (e.g., soft near‑real‑time vs daily; per source).
    • Availability and durability targets.
    • Scalability targets (docs, QPS, tenants).
    • Cost constraints and levers (compute, storage, egress).
  3. Security, Privacy, Compliance
    • Data privacy/PII strategy, encryption in transit/at rest, key management.
    • SOC2/GDPR considerations (data residency, retention, deletion).
    • Tenant isolation (logical and optionally physical).

Indexing Pipeline

Describe the end‑to‑end ingestion and indexing flow:

  1. Connectors and Sync
    • Source connectors (APIs/webhooks/CDC), change detection, incremental sync, backfill.
    • Failure handling, retries, rate limiting, state checkpoints.
  2. Parsing and Enrichment
    • Document parsing (PDF/Office), OCR for images/scans, HTML cleanup, code parsing.
    • Language detection, text normalization/tokenization, boilerplate removal.
    • Metadata extraction (author, created/modified, teams, labels), ACL extraction.
    • PII/sensitive detection and optional redaction/quarantine.
    • Deduplication and near‑duplicate clustering.
    • Chunking strategy (e.g., 512–1,000 tokens with overlap) and parent mapping.
  3. Feature Generation
    • Lexical features (BM25/TF‑IDF, field boosts, bigrams).
    • Semantic features (embeddings for chunks and titles; multilingual models).
    • Auxiliary features (recency, popularity, click priors, authority).

Storage and Retrieval

  1. Storage
    • Inverted index (e.g., Lucene/ES) for lexical retrieval.
    • Vector store (e.g., HNSW) for ANN semantic retrieval.
    • Sharding/partitioning across tenants; metadata and ACL storage.
  2. Hybrid Retrieval Strategy
    • Candidate generation from both lexical and vector stores with ACL‑aware filtering.
    • Score fusion (e.g., weighted blend or Reciprocal Rank Fusion).
    • Candidate sizes and time budgets.

Ranking and Query Understanding

  1. Ranking Stack
    • Options: learning‑to‑rank (e.g., LambdaMART/XGBoost) and/or neural cross‑encoder reranker.
    • Feature set and latency budget by stage.
  2. Query Understanding
    • Spelling correction, synonyms/abbreviations, stemming/lemmatization.
    • Embedding of queries, intent detection, facet suggestion.
    • Multilingual handling (cross‑lingual embeddings, optional translation).

Personalization and Feedback

  • How to incorporate user/org context, click/dwell signals, recency and popularity.
  • Feedback loops: counterfactual learning, bandits, auto‑synonym mining, human‑in‑the‑loop.

Evaluation and Monitoring

  1. Offline Evaluation
    • Metrics: NDCG@k, Recall@k, MRR; dataset curation; train/val/test splits.
  2. Online Evaluation
    • A/B testing, interleaving, guardrails, success KPIs.
  3. Monitoring
    • Latency, errors, zero‑result rate, freshness lag, ANN recall probes, data/feature drift.

Guardrails

  • Sensitive content filters, PII handling, leakage prevention in snippets and counts.
  • Safety, abuse and enumeration defenses; rate limiting.

Scalability and Operations

  • Capacity planning; horizontal scaling; multi‑region DR.
  • Caching layers (query/results, embeddings), invalidation.
  • Result deduplication and grouping.
  • Multilingual support details (tokenization, CJK, right‑to‑left).

Access Control Enforcement

  • Exact plan for enforcing ACLs at query time across lexical and vector retrieval without leaking unauthorized documents.
  • Side‑channel protections (counts, snippets, suggestions).

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More ML System Design•More OpenAI•More Machine Learning Engineer•OpenAI Machine Learning Engineer•OpenAI ML System Design•Machine Learning Engineer ML System Design