Design ML system for self-driving perception
Company: Google
Role: Machine Learning Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Technical Screen
You are interviewing for a Senior Machine Learning Engineer role on a self-driving car team. They ask you to design a machine learning system for **obstacle detection and collision avoidance** in urban driving.
The system should run **on-vehicle** and must:
- Take input from multiple sensors (e.g., cameras, LiDAR, radar).
- Detect relevant objects (cars, pedestrians, cyclists, traffic cones, etc.) and free space.
- Support real-time decision-making to help avoid collisions.
Latency and reliability requirements (you may refine/adjust them during clarifying questions):
- End-to-end perception latency: **≤ 100 ms** per frame.
- Very high recall on obstacles within 50 meters in front of the car.
- System must be robust to weather, lighting, and rare edge cases.
**Design a complete ML system**, covering at least:
1. **Problem definition and metrics**
- What exactly the system should output.
- Offline and online metrics you would use.
2. **Data and labeling**
- What data you will collect from the fleet.
- How you will label obstacles, free space, and other relevant signals.
- How you will handle class imbalance and rare events.
3. **Modeling choices**
- What model architectures you would use for perception (e.g., object detection, segmentation, sensor fusion).
- How you would fuse multi-modal sensor information.
4. **Training pipeline**
- Data preprocessing.
- Train/validation/test splitting strategy.
- Handling distribution shift (e.g., new cities, weather conditions).
5. **Serving / inference architecture**
- How models are deployed on the car.
- How you meet latency and reliability constraints.
- Any model compression or optimization techniques.
6. **Monitoring, feedback loop, and continuous improvement**
- How you monitor model quality in production.
- How you detect and mitigate degradation or new failure modes.
- How you use new data to retrain and update models safely.
Walk through your design step by step, explaining trade-offs and key decisions.
Quick Answer: This question evaluates understanding of end-to-end ML system design for on-vehicle obstacle detection and collision avoidance, encompassing sensor fusion, perception models, latency and reliability constraints, data collection and labeling, and production monitoring within the autonomous driving domain.