Hierarchical Reasoning Model

HRM

HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples.

HRM combines H(slow)/L(fast) two modules with different timescales (multiple L steps → 1 H step). HRM avoids premature convergence through hierarchical convergence (L converges locally → H updates context and resets L). HRM uses 1-step gradient (DEQ 1st-order approximation) + deep supervision for O(1) memory backpropagation. In essence, it combines timescale separation, hierarchical convergence, 1-step gradient, and ACT.

Frequency Hierarchy

Two networks f_L and f_H alternately update z_L and z_H. This is based on the assumption of different frequency layers (hierarchy) in the brain. The paper connects this to

Hierarchical Temporal Processing in the brain (fast sensory vs slow reasoning loop).

arxiv.org

https://arxiv.org/pdf/2506.21734

Less is More: Recursive Reasoning with Tiny Networks

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on...

https://arxiv.org/abs/2510.04871

Hierarchical Reasoning Model

HRM

Frequency Hierarchy

Backlinks

Recommendations