Design a production face recognition system
Company: Capital One
Role: Data Scientist
Category: Machine Learning
Difficulty: hard
Interview Round: Onsite
Design an on-device face-recognition system for mobile access control serving 50M monthly active users with intermittent connectivity. Decide verification vs. identification and justify. Specify model family and embedding dimension, training objective (e.g., ArcFace vs. triplet loss), data scale/augmentation, and evaluation protocol (ROC/DET) with target thresholds (e.g., FAR ≤ 0.001 at TPR ≥ 0.98). Address liveness/anti-spoofing (2D/3D/IR), occlusions (masks/glasses), demographic fairness (threshold calibration across cohorts), privacy (on-device storage, differential privacy, opt-out), and security (template protection, replay attacks). Set p95 latency (<150 ms), memory (<50 MB), and battery constraints for typical mid-tier devices. Choose on-device vs. server inference and describe fallback when offline. Outline monitoring for drift and periodic re-enrollment, and how you would safely A/B test the system (shadow mode, guardrails).
Quick Answer: This question evaluates production ML system design and engineering skills specific to face recognition, covering on-device model selection and embedding design, training objectives and evaluation protocols, anti-spoofing and robustness mechanisms, privacy and template protection, performance/resource constraints, fairness and monitoring, and safe experimentation. It is asked in the Machine Learning domain for Data Scientist roles because it assesses trade-offs between privacy, security, latency, and scalability in real-world systems, and it tests both conceptual understanding of biometric and security principles and practical application for deploying and operating on-device ML.
