Linear Attention

Creator

Creator

Seonglae Cho

Created

Created

2026 Jun 25 16:15

Editor

Editor

Seonglae Cho

Edited

Edited

2026 Oct 30 10:46

Refs

Refs

Linear Attention은 Key와 Value의 outer product를 누적해 메모리를 구성

2020

Transformers are RNNs: Fast Autoregressive Transformers with...

Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences. To...

https://arxiv.org/abs/2006.16236

moonshotai/Kimi-Linear-48B-A3B-Instruct · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct

moonshotai/Kimi-Linear-48B-A3B-Instruct · Hugging Face

Triadic Linear Attention 2026

Linear Attention의 2차원 메모리 상태를 3차원 텐서로 확장해 표현력을 높이는 방법

Triadic Linear Attention: Three-Dimensional Recurrent States for...

Recurrent neural networks (RNNs) compress the historical context into a memory state of fixed size, thus allowing for constant-time inference. The memory state size is a crucial factor in their...

https://arxiv.org/abs/2609.36529

Recommendations

//////////