Design an image detection system
Overview
This question evaluates system design and machine-learning engineering competencies for building a multi-tenant image object-detection service, covering distributed systems, MLOps, model serving, data and model versioning, and computer vision considerations in the ML System Design category, and it focuses on practical application with high-level conceptual architectural reasoning. It is commonly asked to assess the ability to balance trade-offs in scalability, latency, accuracy, cost, observability, privacy, and operational resilience while demonstrating understanding of API flows, performance engineering, monitoring, deployment strategies, and model evaluation.
System Design: End-to-End Image Object-Detection Service
Context
You are designing a multi-tenant cloud service that ingests user images, runs object detection, and serves results via APIs to web/mobile clients and internal services. The system must support both synchronous (request-response) and asynchronous (batch) inference, and be safe, observable, and cost-efficient.
Assume typical production constraints (public cloud, containerized services, GPU-backed model serving) and that you must propose reasonable SLAs/SLOs and scale assumptions. State any additional assumptions you make.
Tasks
Design the system and cover the following:
- Requirements
- Functional: APIs (sync/async), authentication/authorization, idempotency, versioned results, retries, multi-tenancy.
- Non-functional: propose accuracy targets, latency budget (p50/p95), throughput (RPS), availability, durability, and cost goals.
- Architecture
- High-level components: ingestion/gateway, storage (hot/cold), preprocessing, model serving, asynchronous workers/queues, result store, metadata.
- Request flows: synchronous request-response and asynchronous job-based flows.
- Data and Version Management
- Dataset/versioning strategy for training data and labels.
- Model registry, version pinning, schema evolution.
- Performance Engineering
- Batching strategy, GPU utilization tactics, autoscaling (CPU/GPU), and caching layers.
- Modeling Choices
- Single-stage vs. two-stage detectors: trade-offs and when to use each. Quantization/compilation options.
- ML Pipeline
- Training/labeling/QA workflow; augmentation; class imbalance handling.
- Evaluation and Monitoring
- Offline metrics (e.g., mAP variants) and test sets.
- Online monitoring (latency, correctness proxies, drift), alerting, SLIs/SLOs.
- A/B testing or shadow testing strategy.
- Resilience and Operations
- Failure modes, backpressure, retries, idempotency, DLQs.
- Privacy/compliance, data retention, encryption, and access controls.
- Cost controls and capacity planning.
- Rollback strategy, blue/green or canary deployments.
Deliver a concise but complete design, with diagrams-as-text or clear component lists, and include small numeric examples where helpful.
Solution
Login required