How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a hard difficulty Machine Learning question, commonly asked during Technical Screen rounds at Google.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Google during technical interviews.

Build and evaluate illegal-video classifier

Quick Overview

This question evaluates competency in end-to-end Machine Learning system design, including multimodal modeling (vision, audio, text), data engineering for sparse, noisy, and imbalanced labels, robustness and abuse resistance, human-in-the-loop workflows, privacy/retention concerns, and operational metrics.

End-to-End ML System Design: Flag Illegal YouTube Videos

You are tasked with designing a production ML system to detect and triage potentially illegal YouTube videos at scale. The system must work across modalities (vision, audio, text), handle sparse/noisy labels, strong class imbalance, evolving policies, and integrate with human review.

Assumptions (make minimal, explicit):

"Illegal" follows platform policy (e.g., child safety, terror content, incitement to violence), with versioned policy definitions that evolve over time.
Actions include: automatic block, downrank/age-restrict, route to human review, or allow.
The system must support multilingual/global content and near-real-time decisions.

Design the system across the following areas:

1) Data

Inputs: video frames/thumbnails, audio tracks, ASR captions/transcripts, titles/descriptions/tags, uploader/channel metadata, user flags, policy takedown logs.
Constraints: sparse and noisy labels, severe class imbalance, evolving policies.
Describe data ingestion, feature storage, deduplication/near-duplicate handling, label pipelines (including policy-version tracking), and privacy/retention considerations.

2) Modeling

Choose architectures per modality (vision, audio, text) and a multimodal fusion approach.
Pretraining/embeddings strategy (self-supervised/foundation models; multilingual coverage).
Strategy for weak supervision (heuristics, user flags, external lists) and active learning to acquire high-value labels.
Handling class imbalance, noisy labels, and continual learning under policy drift.

3) Evaluation

Offline metrics: AUROC, PR-AUC (class imbalance), calibration (ECE/Brier), and cost-weighted utility.
Thresholding for triage tiers (auto-block, send-to-review, allow), grounded in expected utility and reviewer capacity.
Build a reliable test set that resists leakage, near-duplicates, and distribution shift; include slice-based evaluation (language, region, topic, channel age).

4) Safety and Abuse Resistance

Anticipate adversarial evasion and propose robustification and monitoring (without revealing evasion recipes).
Fairness and false-positive harm mitigation; transparent appeals workflow; reversibility of actions.
Human-in-the-loop design: reviewer tooling, quality control, throughput/SLA constraints, and prioritization.

5) Online Rollout and Measurement

Rollout plan: shadow mode, canary, progressive ramp, and interleaving with existing human/rule systems; kill switches.
Counterfactual risk estimation using IPS/DR to estimate violation risks and action costs offline.
Experiment design to measure reduction in policy violations without selection bias; randomized auditing to estimate true prevalence.

Quick Overview

End-to-End ML System Design: Flag Illegal YouTube Videos

Assumptions (make minimal, explicit):

"Illegal" follows platform policy (e.g., child safety, terror content, incitement to violence), with versioned policy definitions that evolve over time.

Actions include: automatic block, downrank/age-restrict, route to human review, or allow.

The system must support multilingual/global content and near-real-time decisions.

Design the system across the following areas:

1) Data

Inputs: video frames/thumbnails, audio tracks, ASR captions/transcripts, titles/descriptions/tags, uploader/channel metadata, user flags, policy takedown logs.

Constraints: sparse and noisy labels, severe class imbalance, evolving policies.

Describe data ingestion, feature storage, deduplication/near-duplicate handling, label pipelines (including policy-version tracking), and privacy/retention considerations.

2) Modeling

Choose architectures per modality (vision, audio, text) and a multimodal fusion approach.

Pretraining/embeddings strategy (self-supervised/foundation models; multilingual coverage).

Strategy for weak supervision (heuristics, user flags, external lists) and active learning to acquire high-value labels.

Handling class imbalance, noisy labels, and continual learning under policy drift.

3) Evaluation

Offline metrics: AUROC, PR-AUC (class imbalance), calibration (ECE/Brier), and cost-weighted utility.

Thresholding for triage tiers (auto-block, send-to-review, allow), grounded in expected utility and reviewer capacity.

Build a reliable test set that resists leakage, near-duplicates, and distribution shift; include slice-based evaluation (language, region, topic, channel age).

4) Safety and Abuse Resistance

Anticipate adversarial evasion and propose robustification and monitoring (without revealing evasion recipes).

Fairness and false-positive harm mitigation; transparent appeals workflow; reversibility of actions.

Human-in-the-loop design: reviewer tooling, quality control, throughput/SLA constraints, and prioritization.

5) Online Rollout and Measurement

Rollout plan: shadow mode, canary, progressive ramp, and interleaving with existing human/rule systems; kill switches.

Counterfactual risk estimation using IPS/DR to estimate violation risks and action costs offline.

Experiment design to measure reduction in policy violations without selection bias; randomized auditing to estimate true prevalence.

Build and evaluate illegal-video classifier

Quick Overview

Build and evaluate illegal-video classifier

End-to-End ML System Design: Flag Illegal YouTube Videos

1) Data

2) Modeling

3) Evaluation

4) Safety and Abuse Resistance

5) Online Rollout and Measurement

Write your answer

Build and evaluate illegal-video classifier

Quick Overview

Build and evaluate illegal-video classifier

End-to-End ML System Design: Flag Illegal YouTube Videos

1) Data

2) Modeling

3) Evaluation

4) Safety and Abuse Resistance

5) Online Rollout and Measurement

Write your answer