Seonglae ChoAbstractive Non-Deterministic Turing Machine with probability
Autoregression is a statistical technique that uses past values to predict future values in a time series. It's a regression of a variable against itself.
The confusing terminology is that Masked Attention is considered opposite to Bidirectional LM because information about tokens after the current token is not considered. The key point is unidirectionality/bidirectionality
Autoregressive Model Notion
Maxime Labonne - Decoding Strategies in Large Language Models
A Guide to Text Generation From Beam Search to Nucleus Sampling
https://mlabonne.github.io/blog/posts/2022-06-07-Decoding_strategies.html
Decoder-only LLM to Encoder-Decoder Transformer
arxiv.org
https://arxiv.org/pdf/2504.06225
Injective fucntion to final hidden state
arxiv.org
https://arxiv.org/pdf/2510.15511