Explain Decoder-Only LLM Architecture and How vLLM Serves It Efficiently
Company: Amazon
Role: Software Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
Overview: A two-part conversational question for AI infrastructure roles: explain how a decoder-only transformer turns a prompt into next-token probabilities, then explain how vLLM serves such models efficiently. It tests transformer internals, KV cache sizing, prefill versus decode bottlenecks, paged KV memory and continuous batching.
