Design autonomous-driving experience metrics
Quick Overview
This question evaluates a data scientist's competency in designing a metric framework for autonomous-driving rider experience and performing diagnostic analyses on trip-level telemetry and feedback, covering north-star metric selection, safety/comfort/reliability/efficiency guardrails, and detection of map-ETA bias, route mix, selection bias, and data quality issues. It falls under Analytics & Experimentation for a Data Scientist role and is commonly asked because it probes both conceptual understanding of metric trade-offs and practical application of segmentation, diagnostics, and operational validation.
WeRide
Jan 23, 2025, 12:00 AM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0
You are evaluating rider experience for an autonomous-driving ride service. Suppose you have trip-level data with fields such as:
-
trip_id,city,route_type,distance_km -
planned_eta_minfrom a map provider -
actual_ride_time_min -
hard_brake_count,max_jerk,takeover_count,safety_event_flag -
rider_rating,complaint_flag,trip_completed_flag -
traffic_level,weather,time_of_day
- Propose a metric framework for measuring the overall rider experience of the autonomous-driving product. What would you choose as the north-star metric, and what safety, comfort, reliability, and efficiency guardrails would you track alongside it? How would you handle trade-offs between metrics and avoid a misleading single-number summary?
-
One candidate metric is
trip_time_ratio = actual_ride_time_min / planned_eta_min. You find that this ratio is much higher than expected, suggesting poor performance. Describe how you would investigate whether the issue reflects a true product problem versus an artifact of metric definition, route mix, map-ETA bias, selection bias, or data quality problems. Be explicit about segmentations, comparisons, and follow-up analyses.
Solution
Show