Use the Roofline Model to Analyze Prefill and Decoding

Use the roofline model to reason about an LLM inference workload. How do you determine whether a kernel is memory-bound or compute-bound, and how do prefill and decoding differ? ### Constraints & Assumptions Specify the memory level and operation-count convention used in arithmetic intensity. Treat prefill/decoding bottlenecks as workload-dependent rather than universal labels. ### Clarifying Questions What are the batch size, sequence length, precision, hardware compute ceiling, and sustainable memory bandwidth? Are measurements kernel-local or end-to-end? Which bytes are actually transferred? ### What a Strong Answer Covers The roofline equation, arithmetic intensity, measured utilization, data reuse, and the effect of batching and KV reads on inference phases. ### Follow-up Questions Can a kernel sit far below both roofs? How do kernel launches, synchronization, and small shapes affect the diagnosis? What changes when decode batch size or context length grows?