LM Context Extending

Creator

Creator

Seonglae Cho

Created

Created

2023 Jul 9 12:44

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Mar 2 6:3

Refs

Refs

Sparse Attention

Relative Positional Encoding

While long context LLMs offer simplicity, factual recall can be challenged by both context length.

Language Model Context Extending Methods

https://arxiv.org/pdf/2309.16039.pdf

Paper page - Extending Context Window of Large Language Models via Positional Interpolation

Join the discussion on this paper page

Paper page - Extending Context Window of Large Language Models via Positional Interpolation

https://huggingface.co/papers/2306.15595

Paper page - Extending Context Window of Large Language Models via Positional Interpolation

Recommendations

////////