System Design: Hierarchical Multi-Label Classifier for Noisy Taxonomy

Context

You have a catalog of items with hierarchical tags (e.g., Category → Subcategory → Leaf). Tags are:

• Not mutually exclusive (an item can belong to multiple leaves/paths).
• Inconsistent across levels (naming, missing parents, duplicate/overlapping nodes).

Design a production-ready classifier that predicts consistent hierarchical labels for new items, given raw item data (e.g., title, description, images, structured attributes).

Requirements

1. Clarify and define the label space (multi-label vs. multi-class) and decision about predicting leaves vs. all ancestors.
2. Propose data cleaning and taxonomy normalization steps (deduplication, synonym mapping, cycle detection, DAG enforcement, multi-parent handling).
3. Choose model architecture(s) that capture label dependencies (e.g., top-down, multi-task per level, label-graph models) and explain trade-offs.
4. Specify loss functions (binary cross-entropy) and any hierarchical/constraint-aware losses to enforce parent–child consistency and capture co-occurrences.
5. Define thresholding, calibration, and decoding to turn scores into a valid hierarchical set (e.g., per-label thresholds, hierarchical closure, beam search).
6. Handle severe class imbalance and long-tail labels.
7. Propose evaluation metrics at leaf and hierarchy levels (micro/macro F1, PR-AUC, hierarchical precision/recall, path metrics) and how to construct validation splits.
8. Explain training data requirements and strategies to obtain labels at scale (weak supervision, semi-supervised, active learning, PU-learning).
9. Set realistic inference latency/throughput targets and model size constraints, plus optimization tactics.
10. Monitoring and maintenance: data/label drift, calibration, constraint violations, taxonomy updates, human-in-the-loop.