Compare Driving Algorithms Using Hard-Braking Rates
Quick Overview
Design an exposure-aware experiment comparing hard-braking rates across two driving algorithms, with city and road-condition controls and count-data inference.
Compare Driving Algorithms Using Hard-Braking Rates
Company: Waymo
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
You are comparing two autonomous-driving algorithms. The available table records algorithm, city, road-condition category, miles driven, and hard-braking count. For this exercise, comfort is determined only by hard braking: fewer hard-braking events for comparable driving exposure indicate greater comfort.
Design an experiment and a statistical analysis to compare the algorithms. Explain how mileage, city, road conditions, and the distribution of hard-braking counts affect your design, and identify what the existing table cannot establish by itself.
### Constraints and Clarifying Questions
- Clarify what one row represents: a trip, a vehicle over a period, or an aggregate of multiple drives.
- Define hard braking consistently across algorithms and decide whether miles are comparable measures of exposure under different road conditions.
- Clarify whether algorithms were assigned randomly and whether the same vehicles or routes contribute repeated observations.
- No sample size, count distribution, improvement target, or assignment mechanism is supplied. State any statistical model as a hypothesis to check.
- The task is to design and analyze an experiment; a database query alone does not answer it.
```hint Keep the exposure with the count
Two drives can have the same number of hard-braking events but very different mileage. Also consider what happens when an algorithm receives most of the difficult-road mileage.
```
### What a Strong Answer Covers
- A comfort metric, estimand, and comparison direction that account for driving exposure.
- An assignment plan and analysis that address city and road-condition differences without confusing stratification with randomization.
- A suitable starting model for count data, diagnostics for overdispersion or excess zeros, and inference that respects repeated observations.
- An effect estimate and uncertainty interval, plus limitations of observational or aggregated data.
### Follow-up Questions
- What would make a simple Poisson model inappropriate for these counts?
- How would your conclusion change if one algorithm were tested mostly in easier road conditions?
- What additional identifiers would you collect if rows from the same vehicle were correlated?
- How would you plan the experiment when hard-braking events are rare and many rows contain zero events?
Overview: Design an exposure-aware experiment comparing hard-braking rates across two driving algorithms, with city and road-condition controls and count-data inference.
You are comparing two autonomous-driving algorithms. The available table records algorithm, city, road-condition category, miles driven, and hard-braking count. For this exercise, comfort is determined only by hard braking: fewer hard-braking events for comparable driving exposure indicate greater comfort.
Design an experiment and a statistical analysis to compare the algorithms. Explain how mileage, city, road conditions, and the distribution of hard-braking counts affect your design, and identify what the existing table cannot establish by itself.
Constraints and Clarifying Questions
Clarify what one row represents: a trip, a vehicle over a period, or an aggregate of multiple drives.
Define hard braking consistently across algorithms and decide whether miles are comparable measures of exposure under different road conditions.
Clarify whether algorithms were assigned randomly and whether the same vehicles or routes contribute repeated observations.
No sample size, count distribution, improvement target, or assignment mechanism is supplied. State any statistical model as a hypothesis to check.
The task is to design and analyze an experiment; a database query alone does not answer it.
What a Strong Answer Covers Guidance
A comfort metric, estimand, and comparison direction that account for driving exposure.
An assignment plan and analysis that address city and road-condition differences without confusing stratification with randomization.
A suitable starting model for count data, diagnostics for overdispersion or excess zeros, and inference that respects repeated observations.
An effect estimate and uncertainty interval, plus limitations of observational or aggregated data.
Follow-up Questions Guidance
What would make a simple Poisson model inappropriate for these counts?
How would your conclusion change if one algorithm were tested mostly in easier road conditions?
What additional identifiers would you collect if rows from the same vehicle were correlated?
How would you plan the experiment when hard-braking events are rare and many rows contain zero events?