Optimize Tensor Runtime Kernels
Company: Waymo
Role: Machine Learning Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Technical Screen
You are responsible for optimizing an ML framework runtime on an accelerator. A model has high latency and occasional memory pressure. Explain how you would profile and optimize tensor kernels, memory layout, operator scheduling, and kernel fusion. Include tradeoffs between throughput, latency, memory usage, numerical correctness, and maintainability.
Overview: This question evaluates competency in optimizing machine learning framework runtimes for accelerators, covering profiling and performance analysis of tensor kernels, memory layout design, operator scheduling and kernel fusion while weighing trade-offs among throughput, latency, memory usage, numerical correctness, and maintainability.