Seonglae ChoThe output space is a hyperbolic space (Poincaré ball)
Advantageous for representing tree/hierarchical structures with exponentially growing space
- Linear projection:
- Bounded scaling (prevents explosion)
- Exponential map to hyperbolic space:
- Distance calculation using Poincaré distance:
Hyperbolic Geometry of Reasoning: Probing LLM Hidden States
Large language models with chain-of-thought reasoning exhibit hierarchical dependencies, yet the geometric structure of these representations remains underexplored. We probe DeepSeek-R1...
https://openreview.net/forum?id=JmWG0P9MDf