SpaceX New Grad Research Scientist Interview Experience — Speculative Decoding in a 60-Minute ML Coding Round

There were two 60-minute interviews: an ML coding round and a research chat.

The ML coding round asked me to implement speculative decoding. I needed to write one normal decoding function and one speculative decoding function. The speculative function had the following interface in the prompt:

def speculative_decoing(model, tokenizer, prompt, draft, max_tokens)

The test case required the speculative-decoding implementation to produce its output faster than the normal-decoding implementation.