Seonglae ChoWhen vectors are randomly placed in high-dimensional space, they tend to be mostly orthogonal to each other, which may not accurately reflect their semantic relationships
The Linear Representation Hypothesis and the Geometry of Large...
Informally, the 'linear representation hypothesis' is the idea that high-level concepts are represented linearly as directions in some representation space. In this paper, we address two closely...
https://openreview.net/forum?id=T0PoOJg8cK¬eId=hIzVPo3wws