How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a medium difficulty Machine Learning question, commonly asked during Technical Screen rounds at Point72.

What role is this question designed for?

This question is commonly asked for Machine Learning Engineer candidates at Point72 during technical interviews.

Explain Transformer Encoder and Decoder Behavior

Last updated: May 2, 2026

Quick Overview

This question evaluates knowledge of Transformer architecture and generative language modeling, specifically the encoder versus decoder roles, attention patterns and causal masking, plus mechanisms behind stochastic text generation such as sampling and temperature.

|Home/Machine Learning/Point72

Explain Transformer Encoder and Decoder Behavior

Point72

Mar 6, 2026, 12:00 AM

mediumMachine Learning EngineerTechnical ScreenMachine Learning

Answer the following Transformer fundamentals questions in a machine learning interview:

What are the main differences between a Transformer encoder and a Transformer decoder? Your answer should discuss attention patterns and the role of a causal mask.
For the same input prompt, why can an autoregressive decoder-based language model produce different outputs on different runs? Discuss sampling, temperature, and when generation would be deterministic.

Loading comments...

Browse More Questions

More Machine Learning•More Point72•More Machine Learning Engineer•Point72 Machine Learning Engineer•Point72 Machine Learning•Machine Learning Engineer Machine Learning

Write your answer

Your first approved answer each day earns 20 XP.