Describe past NLP work and collaboration

## How to structure strong answers Use a consistent framework so you don’t ramble: - **STAR**: Situation → Task → Action → Result - Add **“Reflection”** at the end: what you learned / what you’d do differently - Keep a **clear “you vs team”** boundary: “I owned…”, “I collaborated on…”, “The team decided…” Where possible, quantify results: - Model metrics: accuracy/F1/AUROC, calibration, latency, cost - Data metrics: label quality (IAA), disagreement rate, coverage, drift - Product metrics: CTR, conversion, user satisfaction, reduced ops time --- ## 1) Explaining a protocol from your resume ### What the interviewer is really testing - Can you communicate technical concepts clearly to a non-specialist? - Do you understand fundamentals vs. memorizing buzzwords? - Did you actually contribute, and at what depth? ### A good outline (2–4 minutes) 1. **One-liner definition:** What problem the protocol solves. 2. **Actors and flow:** Who talks to whom; what messages/states exist. 3. **Key properties:** e.g., reliability, ordering, security, consistency, idempotency. 4. **Trade-offs:** e.g., latency vs. consistency; overhead vs. robustness. 5. **Your contribution:** Design decisions, implementation, debugging, rollout, metrics. ### Example phrasing template - “At a high level, X protocol is used to ____. The main participants are ____. The typical flow is ____. The tricky parts are ____ (e.g., retries, timeouts, ordering). We chose it over alternatives because ____. I personally owned ____ and validated it by measuring ____.” ### Common pitfalls - Giving a Wikipedia definition without connecting to your system. - Not stating constraints (scale, latency, failure modes, threat model). - Claiming ownership without evidence (no details, no metrics, no incidents). --- ## 2) Tricky NLP problem: method + why ### What the interviewer is really testing - Problem formulation: classification vs. ranking vs. generation vs. sequence labeling. - Data realism: noisy labels, imbalance, multilingual, domain shift, long-tail. - Experimental discipline: baselines, ablations, offline/online metrics. - Practical trade-offs: inference cost, latency, interpretability, safety. ### Recommended answer structure **S/T (set the stage):** - What was the business/user goal? - What made it “tricky”? Pick 1–2 concrete reasons: - ambiguous language / sarcasm / code-switching - long-tail entities - label noise and low agreement - domain shift (train vs. production) - privacy constraints / limited data **A (what you did):** 1. **Baseline first:** simple model + simple features; establish a bar. 2. **Data work:** cleaning, taxonomy, sampling, augmentation, label guidelines. 3. **Modeling choice:** e.g., fine-tuning a transformer, CRF head, retrieval-augmented approach, distillation for latency. 4. **Why this method:** connect to constraints. - If low data: transfer learning, parameter-efficient tuning (LoRA), weak supervision. - If label noise: robust loss, filtering, re-annotation, confidence learning. - If long-tail: class-balanced loss, focal loss, curated hard negatives. 5. **Evaluation plan:** - offline metric aligned to goal (e.g., macro-F1 for imbalance) - error analysis slices (language, region, entity types) - calibration and thresholds if it’s a decision system **R (results):** - Provide numbers and impact: “macro-F1 +6 points”, “reduced false positives by 20%”, “latency < 50ms p95”, “annotation cost down 30%”. **Reflection:** - “The biggest lesson was ____; next time I’d ____.” ### Mini checklist: “Why this method?” (make it explicit) - **Constraint** → **Design choice** mapping, e.g.: - “Need low latency” → distillation/quantization - “Need interpretability” → simpler model + explanations + calibrated thresholds - “High ambiguity” → better labeling schema + multi-label + uncertainty handling ### Pitfalls to avoid - Only talking about the model, not the data. - No baselines/ablations. - Using the wrong metric (e.g., accuracy with heavy imbalance). --- ## 3) Working with annotators: challenges and how you handled them ### What the interviewer is really testing - Can you operationalize ML data quality? - Cross-functional communication and empathy. - Process design: guidelines, QA, feedback loops, disagreement resolution. ### Strong answer ingredients 1. **Annotation goal and schema:** What labels, what definitions, what edge cases. 2. **Guidelines & training:** Examples, counterexamples, decision trees. 3. **Quality measurement:** - inter-annotator agreement (Cohen’s κ / Krippendorff’s α) - gold set / audit sampling - adjudication process 4. **Disagreement handling:** - clarify definitions, add rules - add “uncertain/other” bucket when appropriate - escalation path to domain expert 5. **Feedback loop:** - weekly calibration sessions - track top confusion pairs and update guidelines 6. **Throughput vs. quality trade-off:** What SLA existed and how you balanced. ### Common real-world challenges (pick the ones that match your story) - Ambiguous cases leading to low agreement - Annotators optimizing for speed over quality - Drift in guidelines over time - Cultural/language differences affecting interpretation - Difficult edge cases and evolving taxonomy ### Example metrics you can cite - “Agreement improved from κ=0.42 to κ=0.65 after guideline revision and calibration.” - “Audit error rate dropped from 12% to 5%.” - “We reduced rework by 30% by introducing a gold set and adjudication.” ### Pitfalls - Blaming annotators instead of improving the process. - No measurable quality control. --- ## Quick preparation tips - Prepare **3 stories** that cover: technical depth, ambiguity, collaboration/conflict. - For each story, write down: goal, constraints, what you did, metrics, and a lesson learned. - Have a 30-second and a 2-minute version of each answer.