Seonglae ChoIt uses a structure that dynamically updates a fixed-length “memory” while processing the input in segments. Inspired by how we take notes and compress key information when reading long documents, MemAgent is an agent-based approach that can process arbitrarily long text with linear complexity ($O(n)$). The model actively overwrites and updates its memory (Overwrite) as a strategy.
MemAgent: Reshaping Long-Context LLM with Multi-Conv RL based Memory Agent
https://memagent-sialab.github.io/
MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation...
https://arxiv.org/abs/2507.02259
