Describe internship and research projects
Company: TikTok
Role: Machine Learning Engineer
Category: Behavioral & Leadership
Difficulty: medium
Interview Round: Technical Screen
Briefly introduce two projects—one internship and one research. For each, state the problem, your role and ownership, key technical decisions and why, notable challenges and how you addressed them, measurable results/impact, and one improvement you would make if you had more time.
Quick Answer: This question evaluates end-to-end ownership, leadership and communication skills alongside technical depth in machine learning systems, including model selection, data pipelines, performance metrics, and scalability.
Solution
## How to Answer (Template)
Use a tight STAR+R structure (Situation, Task, Action, Result, Reflection):
- Situation/Problem: Who is the user? What metric matters? What constraints exist (e.g., p99 latency ≤ 50 ms, 10M DAU, 1% positives)?
- Role/Ownership: What did you own end-to-end? What decisions were yours vs. the team’s?
- Actions/Key Decisions: Models, features, metrics, infra; explain trade-offs and why you chose them.
- Challenges/Fixes: Data leakage, skew, latency, cost, instability; how you diagnosed and resolved them.
- Results/Impact: Offline (AUC, AUCPR, loss), online (CTR, watch time, conversion), reliability (p99, QPS), and business outcomes. Include numbers.
- Improvement: The most leveraged next step (e.g., sequence model, bandits, better labeling, caching).
Tip: Tie offline to online. Call out constraints (latency, memory, privacy). Mention guardrails (holdouts, canaries, power).
---
## Example 1 — Internship: Real‑Time Feed Ranking Upgrade
- Problem
- Goal: Increase short‑video feed engagement (CTR and watch time) without exceeding a 50 ms p99 inference budget for ranking.
- Scale: ~30M daily ranking requests; mixed device types.
- Role & Ownership
- ML Engineer intern. Owned candidate feature pipeline and ranking model upgrade from baseline logistic regression to a low‑latency gradient‑boosted tree model.
- Led offline evaluation, feature ablations, and partnered with platform team for serving optimizations.
- Key Technical Decisions and Why
1) Model: Chose XGBoost with monotonic constraints over LR and over deeper DNNs.
- Why: Captures nonlinearities for CTR while maintaining predictable low latency on CPU; monotonic constraints stabilized business‑critical features (e.g., prior views → non‑decreasing CTR).
2) Features: Added sequence‑aware aggregates (e.g., last‑N engagement rates), time‑decayed counts, and content‑creator embeddings.
- Why: Recent behavior often predicts short‑term interest; decay reduces drift; embeddings generalize to sparse IDs.
3) Data & Splits: Time‑based splits; K‑fold target encoding with out‑of‑fold leakage prevention.
- Why: Prevents future→past leakage and overfitting via naïve encodings.
4) Serving: Feature store with feature parity checks; model quantization (INT8) + ONNX Runtime; micro‑batching for CPU vectorization.
- Why: Match training/serving features; meet p99 latency and reduce cost.
5) Metrics: Optimized for calibrated pCTR and Precision@K; business eval on CTR and avg watch time.
- Why: Ranking quality aligns with surface CTR/watch time; calibration helps auction‑style blending.
- Challenges and Fixes
- Training–serving skew: Mismatch in time windows caused offline–online drift.
- Fix: Feature contracts in the feature store, unit tests on window bounds; CI to diff stats (mean/std, missing rate) between offline and online.
- p99 latency regression (63 ms) after feature expansion.
- Fix: Quantization + cache hot features; pruned low‑gain trees; achieved 44 ms p99.
- Cold‑start items/users.
- Fix: Back‑off features using content metadata and creator priors; hashing for unseen IDs; calibrated fallback scores.
- Results/Impact
- Offline: AUC +0.018; Calibrated ECE reduced from 0.045 → 0.021.
- Online A/B (2 weeks, 50/50, CUPED): CTR +2.1% (p<0.05), avg watch time/session +1.3%, no increase in bounce; p99 44 ms (−12% vs baseline) and −18% CPU per 1k requests.
- One Improvement
- Sequence model (DIN/DIEN or lightweight Transformer) for user–item interactions under a 50 ms budget using distillation to a tree or a small MLP; expected +0.5–1.0% CTR with careful caching.
- Guardrails Used
- Time‑based split; canary rollout (5%) before 50/50; kill‑switch on p99 > 55 ms; feature drift alerts.
- Small Numeric Example: A/B Power Check
- Baseline CTR p0 = 5.0%, target uplift δ = +0.10 pp (2% relative). For two‑sided α=0.05, 1−β=0.8:
- n per group ≈ 2 × (z_{0.975}+z_{0.8})^2 × p0(1−p0) / δ^2 ≈ 2 × (1.96+0.84)^2 × 0.05×0.95 / 0.001^2 ≈ 25.5M impressions per arm.
- Ensures observed +0.1 pp is detectable.
---
## Example 2 — Research: Semi‑Supervised Multimodal Content Moderation
- Problem
- Goal: Detect policy‑violating content in short videos with limited labeled data and heavy class imbalance (~1% positives). Prior system had poor recall at low FPR, causing reviewer overload.
- Constraints: Keep FPR ≤ 1% to avoid creator friction; support 100+ videos/sec/GPU.
- Role & Ownership
- Lead graduate researcher. Designed training objective, multimodal architecture, and labeling strategy; built the pipeline, ran ablations, and led offline ↔ online validation.
- Key Technical Decisions and Why
1) Semi‑Supervised Learning: Mean‑Teacher consistency + pseudo‑labels with confidence threshold τ=0.9.
- Why: Leverage millions of unlabeled videos; teacher EMA stabilizes training; high‑confidence pseudo‑labels mitigate noise.
2) Losses: Focal loss for labeled data to handle imbalance; consistency loss for unlabeled.
- Focal loss: FL(p_t) = −α (1−p_t)^γ log(p_t), used γ=2, α tuned via validation.
- Why: Emphasizes hard positives/negatives; improves AUCPR under skew.
3) Multimodal Late Fusion: Text (ASR/transcript) → transformer encoder; vision → EfficientNet; audio → CNN; stacked with calibration (isotonic).
- Why: Different modalities catch different violations; late fusion is robust to missing modalities.
4) Metrics: Optimized AUCPR offline; selected operating points by maximizing recall at FPR=1%.
- Why: AUCPR reflects performance under imbalance; operations need stable FPR.
5) Robustness: Strong augmentations (SpecAugment for audio, RandAugment for vision), and label‑noise filtering via small‑loss trick.
- Challenges and Fixes
- Label noise and spurious correlations (e.g., background text).
- Fix: Co‑teaching small‑loss filtering; SHAP audits to remove shortcut features; added text masking augmentation.
- Distribution shift across regions and languages.
- Fix: Domain‑specific batch norm; language‑aware heads; calibrated thresholds per locale.
- Threshold calibration instability.
- Fix: Isotonic regression on a time‑decayed validation set; temperature scaling for each modality prior to fusion.
- Results/Impact
- Offline: AUCPR 0.42 → 0.58; at 1% FPR, recall 60% → 77%.
- Ops impact (pilot, 4 weeks): −18% manual review volume at same precision; reviewer SLA variance −22%.
- Throughput: FP16 inference, 120 vids/sec/GPU; p95 latency 38 ms per video modality pass; graceful degradation if ASR missing.
- One Improvement
- Active learning loop with uncertainty sampling + diversity (k‑center) to label edge cases weekly; expected AUCPR +0.03 with ~5k labels/week. Longer‑term: pretrain with contrastive multimodal learning (CLIP‑style) on 100M public videos.
- Validation/Guardrails
- Strict temporal validation; per‑locale holdouts; fairness slice checks (language/creator size). Shadow deployment before partial automation. Alerting on FPR drift using Wilson intervals.
---
## Pitfalls to Avoid
- Vague ownership (“we did”); always state what you personally built/decided.
- Only offline metrics; always tie to business/user impact and latency/cost.
- Ignoring guardrails (leakage checks, canaries, power analysis, rollback).
- No trade‑offs; explain why not another model (e.g., DNN too slow, ANN retrieval unnecessary, etc.).
Use the template to swap in your own projects; keep each to 6 bullets with crisp numbers and one thoughtful improvement.