AI Spatial feature

Using linear/nonlinear probing and PCA, we discovered a 3D subspace where 'up'↔'down', 'left'↔'right' relationships emerge as orthogonal, opposite direction vectors. Diagonal relationships were precisely composed as sums of those basis vectors, and object position embeddings formed consistent coordinate clusters. In steering experiments, we achieved a 74.3% success rate in manipulating the model's predicted directions. This demonstrates that LLMs internalize interpretable spatial world models from text alone, though further research is needed for temporal changes and more complex relationships.