##### Question You are interviewing for an autonomous-driving metrics team. The team wants to measure the rider experience of completed rides. You have trip-level data with fields such as: - `trip_id`, `city`, `route_type`, `distance_km` - `planned_eta_min` (ETA from a third-party map/navigation provider for the same origin, destination, and request time) - `actual_ride_time_min` - `hard_brake_count`, `max_jerk`, `takeover_count`, `safety_event_flag` - `rider_rating`, `complaint_flag`, `trip_completed_flag` - `traffic_level`, `weather`, `time_of_day` 1. **Metric framework.** Design a metric framework for ride quality. Go beyond a single KPI and cover multiple dimensions: - **efficiency** — total ride time, excess travel time, pickup delay, or detour versus a reference route; - **comfort** — hard braking, jerk, oscillatory behavior, or unnecessary lane changes; - **safety proxies** — critical interventions/takeovers, near-miss indicators, or time-to-collision based measures; - **reliability and trust** — drop-off accuracy, cancellation rate, rider complaints, or route stability. 2. **North-star metric and trade-offs.** What would you choose as the north-star (primary) metric, and what safety, comfort, reliability, and efficiency guardrails would you track alongside it? How would you handle trade-offs between metrics and avoid a misleading single-number summary? 3. **Diagnose a bad ratio.** Consider the candidate metric `eta_ratio = actual_ride_time / baseline_eta` (equivalently `trip_time_ratio = actual_ride_time_min / planned_eta_min`). You find this ratio is unexpectedly high, suggesting the autonomous-driving ride is much slower than the baseline. How would you investigate whether this reflects a *true product problem* versus an artifact of metric definition, bad metric design, route mix, map-ETA bias, selection bias, confounding, or data-quality problems? Be explicit about metric definitions, segmentations, comparisons, possible failure modes, and follow-up analyses. 4. **Additional data.** What additional data would you request to complete the diagnosis?

Design autonomous-ride quality metrics and diagnose a high ETA ratio with safety guardrails, Good Trip Rate, efficiency, comfort, reliability, segmentation, denominator calibration, matched comparisons, delay decomposition, and additional telemetry requests.

How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at WeRide.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at WeRide during technical interviews.

Design Ride-Quality Metrics and Diagnose Ratios

You are interviewing for an autonomous-driving metrics team. The team wants to measure rider experience for completed rides using trip-level data such as city, route type, distance, planned ETA, actual ride time, hard brakes, jerk, takeovers, safety events, rider ratings, complaints, completion status, traffic, weather, and time of day.

Constraints & Assumptions

Safety must be a guardrail and cannot be averaged away by speed or comfort gains.
Avoid relying on one ratio or one aggregate metric.
Normalize and segment metrics by route context, traffic, weather, distance, city, and software version.
Diagnose whether a bad metric reflects a true product problem or a metric/data artifact.

Clarifying Questions to Ask Guidance

Is the metric for executive reporting, model evaluation, release gating, or root-cause diagnosis?
Does actual_ride_time_min include pickup wait, curbside time, or only in-vehicle time?
When and how is planned_eta_min captured?
Do we have matched human-driven or baseline rides for comparison?

Part 1 - Design A Ride-Quality Metric Framework

What metric framework would you build for ride quality?

What This Part Should Cover Guidance

Efficiency, comfort, safety proxies, reliability/trust, and customer sentiment.
Distributional metrics such as median, p90, and p95, not only averages.
Normalization per trip, per kilometer, or per hour where appropriate.

Part 2 - Choose A North Star And Guardrails

What would be the north-star metric, and what guardrails would you track?

What This Part Should Cover Guidance

A primary composite such as Good Trip Rate.
Separate guardrails for safety, comfort, reliability, efficiency, complaints, and completion.
Explanation of why a weighted single number can be misleading.

Part 3 - Diagnose A High ETA Ratio

The candidate metric eta_ratio = actual_ride_time / planned_eta_min is unexpectedly high. How would you investigate?

What This Part Should Cover Guidance

Metric definition validation, data quality checks, segmentation, denominator calibration, confounding control, and delay decomposition.
Absolute delay alongside ratio.
Route mix, map-ETA bias, selection bias, traffic/weather, and AV behavior root causes.

Part 4 - Request Additional Data

What additional data would complete the diagnosis?

What This Part Should Cover Guidance

Planned versus executed route, speed profile, stop reasons, intervention logs, route snapshots, pickup/dropoff timestamps, human baseline, scenario tags, and complaint labels.

What a Strong Answer Covers Guidance

Treats metric design and diagnosis separately.
Protects safety and trust guardrails.
Explains why ETA ratio can be misleading.
Uses matched or adjusted comparisons before declaring a product issue.

Follow-up Questions Guidance

What if the high ratio appears only on short trips?
How would you calibrate third-party map ETA?
What if safety improves while speed worsens?
How would you diagnose whether pickup, routing, or control behavior caused the delay?
What metric would you use for release gating?

Constraints & Assumptions

Safety must be a guardrail and cannot be averaged away by speed or comfort gains.
Avoid relying on one ratio or one aggregate metric.
Normalize and segment metrics by route context, traffic, weather, distance, city, and software version.
Diagnose whether a bad metric reflects a true product problem or a metric/data artifact.

Clarifying Questions to Ask Guidance

Is the metric for executive reporting, model evaluation, release gating, or root-cause diagnosis?
Does actual_ride_time_min include pickup wait, curbside time, or only in-vehicle time?
When and how is planned_eta_min captured?
Do we have matched human-driven or baseline rides for comparison?

Part 1 - Design A Ride-Quality Metric Framework

What metric framework would you build for ride quality?

What This Part Should Cover Guidance

Efficiency, comfort, safety proxies, reliability/trust, and customer sentiment.
Distributional metrics such as median, p90, and p95, not only averages.
Normalization per trip, per kilometer, or per hour where appropriate.

Part 2 - Choose A North Star And Guardrails

What would be the north-star metric, and what guardrails would you track?

What This Part Should Cover Guidance

A primary composite such as Good Trip Rate.
Separate guardrails for safety, comfort, reliability, efficiency, complaints, and completion.
Explanation of why a weighted single number can be misleading.

Part 3 - Diagnose A High ETA Ratio

The candidate metric eta_ratio = actual_ride_time / planned_eta_min is unexpectedly high. How would you investigate?

What This Part Should Cover Guidance

Metric definition validation, data quality checks, segmentation, denominator calibration, confounding control, and delay decomposition.
Absolute delay alongside ratio.
Route mix, map-ETA bias, selection bias, traffic/weather, and AV behavior root causes.

Part 4 - Request Additional Data

What additional data would complete the diagnosis?

What This Part Should Cover Guidance

Planned versus executed route, speed profile, stop reasons, intervention logs, route snapshots, pickup/dropoff timestamps, human baseline, scenario tags, and complaint labels.

What a Strong Answer Covers Guidance

Treats metric design and diagnosis separately.
Protects safety and trust guardrails.
Explains why ETA ratio can be misleading.
Uses matched or adjusted comparisons before declaring a product issue.

Follow-up Questions Guidance

What if the high ratio appears only on short trips?
How would you calibrate third-party map ETA?
What if safety improves while speed worsens?
How would you diagnose whether pickup, routing, or control behavior caused the delay?
What metric would you use for release gating?

Design Ride-Quality Metrics and Diagnose Ratios

Quick Overview

Design Ride-Quality Metrics and Diagnose Ratios

Constraints & Assumptions

Clarifying Questions to Ask Guidance

Part 1 - Design A Ride-Quality Metric Framework

What This Part Should Cover Guidance

Part 2 - Choose A North Star And Guardrails

What This Part Should Cover Guidance

Part 3 - Diagnose A High ETA Ratio

What This Part Should Cover Guidance

Part 4 - Request Additional Data

What This Part Should Cover Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer

Design Ride-Quality Metrics and Diagnose Ratios

Quick Overview

Design Ride-Quality Metrics and Diagnose Ratios

Constraints & Assumptions

Clarifying Questions to Ask Guidance

Part 1 - Design A Ride-Quality Metric Framework

What This Part Should Cover Guidance

Part 2 - Choose A North Star And Guardrails

What This Part Should Cover Guidance

Part 3 - Diagnose A High ETA Ratio

What This Part Should Cover Guidance

Part 4 - Request Additional Data

What This Part Should Cover Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer