Seonglae ChoHRM
HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples.
HRM combines H(slow)/L(fast) two modules with different timescales (multiple L steps → 1 H step). HRM avoids premature convergence through hierarchical convergence (L converges locally → H updates context and resets L). HRM uses 1-step gradient (DEQ 1st-order approximation) + deep supervision for O(1) memory backpropagation. In essence, it combines timescale separation, hierarchical convergence, 1-step gradient, and ACT.
Frequency Hierarchy
Two networks f_L and f_H alternately update z_L and z_H. This is based on the assumption of different frequency layers (hierarchy) in the brain. The paper connects this to Hierarchical Temporal Processing in the brain (fast sensory vs slow reasoning loop).
arxiv.org
https://arxiv.org/pdf/2506.21734
Less is More: Recursive Reasoning with Tiny Networks
Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on...
https://arxiv.org/abs/2510.04871
