Design the manual review workflow
System Design: Human-in-the-Loop Review Subsystem
Context
You are designing a human-in-the-loop (HITL) review subsystem for a large-scale safety platform that moderates user-generated content (UGC) across text, images, and audio (including live voice). Automated detectors (ML models and rules) generate “detections” with metadata (content IDs, model type, confidence, policy category, timestamps). Some detections require immediate enforcement; others need human review for accuracy, context, or policy interpretation.
Requirements
Design and explain the end-to-end HITL subsystem, covering:
-
How review tasks are generated from detections (schema, deduplication, aggregation, idempotency, sampling).
-
Triage into queues by severity with prioritization and dynamic aging.
-
Reviewer UI requirements and ergonomics, including audio-specific needs.
-
SLAs/SLOs per queue and breach handling.
-
Sampling and double-blind consensus to ensure quality; inter-rater agreement.
-
Gold-standard audits (honeypots), reviewer calibration, and performance scoring.
-
Escalation paths and requeueing logic for ambiguous or time-sensitive items.
-
Access control and privacy for sensitive audio and PII.
-
Audit logs and tamper-evident trail.
-
Feedback loop where reviewer decisions update model thresholds, rules, and training data.
-
Capacity planning for reviewers and backlog control during surges.
State reasonable assumptions where necessary and be explicit about trade-offs.
Constraints & Assumptions
-
Preserve the scope, facts, inputs, and requested outputs from the prompt above.
-
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
-
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.
Clarifying Questions to Ask
-
Clarify users, core use cases, read/write patterns, scale, latency, availability, and data retention.
-
State explicit assumptions before making sizing or architecture decisions.
-
Prioritize the functional path first, then address reliability, security, observability, and rollout.
What a Strong Answer Covers
-
A scoped requirements summary with concrete non-goals and success metrics.
-
API, data model, architecture, consistency, capacity, and operations.
-
Reasoned trade-offs among simple and scalable designs, including bottlenecks and failure modes.
-
A validation, monitoring, migration, and launch plan appropriate for the risk level.
Follow-up Questions
-
What breaks first at 10x traffic or data volume?
-
How would you degrade gracefully during dependency failures?
-
What metrics and alerts would prove the design is healthy after launch?