Build an end-to-end ML pipeline

Quick Overview

Build an end-to-end ML pipeline evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Build an end-to-end ML pipeline

Company: Amazon

Role: Machine Learning Engineer

Category: ML System Design

Difficulty: hard

Interview Round: Onsite

Given a CSV with shipment events (order_id, origin, destination, ship_date, promised_date, carrier, weight, item_count, scan_events[], delivered_date), build from scratch a Python pipeline that: ( 1) loads and validates data; handles missing values, outliers, and time zones; ( 2) creates features (e.g., day-of-week, route, carrier stats, dwell times from scan_events); ( 3) labels examples as delayed if delivered_date − promised_date > 48 hours (justify how you handle undelivered items and censoring); ( 4) trains a baseline model (logistic regression or gradient-boosted trees) with cross-validation; reports ROC-AUC and PR-AUC; addresses class imbalance; ( 5) calibrates probabilities and explains top features; ( 6) outputs a CSV of top-K at-risk shipments with calibrated probabilities and reason codes. Optimize for runtime < 5 minutes on 1M rows and memory < 4 GB, and discuss strategies to speed up training/inference and ensure reproducibility.

Quick Answer: Build an end-to-end ML pipeline evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/ML System Design/Amazon
Amazon logo
Amazon
Jul 17, 2025, 12:00 AM
hardMachine Learning EngineerOnsiteML System Design
6
0

Build an end-to-end ML pipeline

ML System Design: Shipment Delay Risk Scoring From a Single CSV

You are given a CSV of shipment events with the following columns:

  • order_id (string)
  • origin (string)
  • destination (string)
  • ship_date (string/datetime)
  • promised_date (string/datetime)
  • carrier (string)
  • weight (float)
  • item_count (int)
  • scan_events (JSON array encoded as string; each element typically has a timestamp and status)
  • delivered_date (string/datetime; may be null if undelivered)

Build a Python pipeline from scratch that:

  1. Loads and validates data, handling missing values, outliers, and time zones.
  2. Creates features (e.g., day-of-week, route, carrier stats via target encoding, and dwell times from scan_events).
  3. Labels examples as delayed if delivered_date − promised_date > 48 hours. Justify and implement how you handle undelivered items and censoring.
  4. Trains a baseline model (logistic regression or gradient-boosted trees) with cross-validation; reports ROC-AUC and PR-AUC; addresses class imbalance.
  5. Calibrates probabilities and explains top features.
  6. Outputs a CSV of top-K at-risk shipments with calibrated probabilities and reason codes.

Constraints:

  • Optimize for runtime < 5 minutes on 1M rows and memory < 4 GB on CPU.
  • Discuss strategies to speed up training/inference and ensure reproducibility.

Clarifying Questions to Ask Guidance

  • Clarify users, core use cases, read/write patterns, scale, latency, availability, and data retention.
  • State explicit assumptions before making sizing or architecture decisions.
  • Prioritize the functional path first, then address reliability, security, observability, and rollout.

What a Strong Answer Covers Guidance

  • A scoped requirements summary with concrete non-goals and success metrics.
  • ML-specific data, model, evaluation, serving, and monitoring choices.
  • Reasoned trade-offs among simple and scalable designs, including bottlenecks and failure modes.
  • A validation, monitoring, migration, and launch plan appropriate for the risk level.

Follow-up Questions Guidance

  • What breaks first at 10x traffic or data volume?
  • How would you degrade gracefully during dependency failures?
  • What metrics and alerts would prove the design is healthy after launch?

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More ML System Design•More Amazon•More Machine Learning Engineer•Amazon Machine Learning Engineer•Amazon ML System Design•Machine Learning Engineer ML System Design