How Autoregressive Models Predict a Sequence From Its Own Past Values
Company: Sierra
Role: Product Manager
Category: Machine Learning
Difficulty: medium
Interview Round: Other
Explain what autoregression is and how an autoregressive model produces its predictions.
```hint Unpack the name
Take the word apart: what is being regressed, and on what? Then follow a single prediction and ask what the model does with the value it has just produced.
```
### Clarifying Questions
- Should I explain autoregression in its classic time-series sense, as the way large language models generate text, or both?
- Should I pitch this to a technical peer, or as I would explain it to non-technical stakeholders on a product team?
- Do you want the math (the model equation and how it is fitted), or mainly the intuition and what it means for a product?
### What a Strong Answer Covers
- A precise definition of what the model regresses on, stated in plain words
- A concrete example, such as a forecasting equation or a sentence being generated step by step
- How the model is trained and how predictions several steps ahead are produced
- Limitations such as compounding errors, sequential latency and reliance on past patterns
- Why the concept matters for product decisions in an AI-powered product
### Follow-up Questions
- Why does an autoregressive language model take longer to answer when its reply is longer, and how would that shape the design of a chat or voice product?
- How does autoregression differ from an ordinary regression on outside features, and when would you combine the two?
- If an AI agent states a wrong fact early in a reply and then builds on it, how does the autoregressive setup explain that, and what would you do about it?
Overview: A concept question for product manager candidates on what autoregression means and how an autoregressive model makes its predictions. It checks whether the candidate can define the idea precisely, illustrate it with forecasting or text generation, and connect its limits to product decisions such as latency, cost and reliability.