Design a scalable video search system
Company: Runway
Role: Machine Learning Engineer
Category: ML System Design
Difficulty: hard
Interview Round: Technical Screen
Design an end-to-end video search system that supports text-to-video and video-to-video retrieval. Specify the model architecture (e.g., dual encoders for video frames/clips and text, temporal aggregation strategies, pretraining/fine-tuning objectives, negative sampling, handling long videos via clip sampling), the shared embedding space, and relevance scoring. For infrastructure, describe the offline/online feature extraction pipeline, ingestion, metadata enrichment, indexing in a vector store (e.g., HNSW/IVF, sharding and replication), storage layout for thumbnails/clips, query routing, latency/cost targets, caching, and scalability for billions of clips. Include monitoring, abuse/content safety, and evaluation via offline metrics and online A/B tests; discuss relevance feedback and incremental model updates.
Quick Answer: This interview question evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer for Design a scalable video search system states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.