Predicting Delivery ETA (Pickup → Drop-off): Case For or Against ML

Context: You’re proposing an ETA model to predict the time from rider pickup to customer drop-off for a last‑mile delivery product. Some stakeholders prefer a “pure analytics” approach. Make a structured case for or against an ML model, including guardrails and alternatives.

1) Target definition and dispatch-time features

• Define the target rigorously, including how you will handle:
  • Cancellations
  • Rider reassignments
  • Multi-stop batches (sequence position, spillover from earlier stops)
• List only features available at dispatch time to avoid leakage.

2) Offline evaluation plan

• Metrics (e.g., MAE, P90 error) across strata (e.g., distance, time-of-day, platform/vehicle type).
• Calibration checks.
• Out-of-time validation to capture seasonality.

3) Online rollout plan

• Guardrail metrics (e.g., cost/order, SLA breaches).
• Shadow vs. interleaved traffic strategy.
• Fallback heuristics during model outage.
• Fairness monitoring across zones.

4) Baseline and threshold for productionization

• Define a simple-but-strong baseline (e.g., segment-aware median).
• State the minimal measurable lift needed to justify engineering cost.

5) If deciding against ML

• Prescribe an analytics-first alternative (e.g., policy changes, pricing/surge rules).
• Provide a decision tree with triggers that revisit ML when thresholds are met.