Design a production face recognition system
Overview
This question evaluates production ML system design and engineering skills specific to face recognition, covering on-device model selection and embedding design, training objectives and evaluation protocols, anti-spoofing and robustness mechanisms, privacy and template protection, performance/resource constraints, fairness and monitoring, and safe experimentation. It is asked in the Machine Learning domain for Data Scientist roles because it assesses trade-offs between privacy, security, latency, and scalability in real-world systems, and it tests both conceptual understanding of biometric and security principles and practical application for deploying and operating on-device ML.
Design an On-Device Face Recognition System for Mobile Access Control
Context
You are designing a face-based access control system for mobile devices with 50M monthly active users. Devices often operate with intermittent connectivity, so the system must work fully offline and sync/update when online. The product must meet strong privacy and security requirements typical for consumer finance and protect against spoofing.
Requirements
- Choose between verification (1:1) and identification (1:N) and justify the choice.
-
Specify:
- Model family and embedding dimension.
- Training objective (e.g., ArcFace vs. triplet loss).
- Data scale and augmentations.
- Evaluation protocol (ROC/DET) and target thresholds (e.g., FAR ≤ 0.001 at TPR ≥ 0.98).
-
Address robustness:
- Liveness/anti-spoofing (2D/3D/IR), replay attacks.
- Occlusions (masks, glasses), low light, motion blur.
- Demographic fairness and threshold calibration across cohorts.
-
Privacy and security:
- On-device storage, template protection, differential privacy, opt-out.
- Secure model updates, integrity, and abuse prevention.
-
Performance constraints for mid-tier devices:
- p95 latency < 150 ms per unlock.
- Memory footprint < 50 MB total.
- Battery impact minimal.
- On-device vs. server inference and offline fallback behavior.
- Monitoring for data drift and periodic re-enrollment.
- Safe A/B testing plan (shadow mode, guardrails).
Solution
Login required