AI Arithmetic Reasoning

In fixed-width text, the model must predict \n by determining whether the next word exceeds the line boundary. The model represents character count (current line length), total line width, remaining character count, next word length, etc. on a 1-dimensional "feature

Manifold". This manifold takes a

Helix shape embedded in a low-rank subspace (≈6D) of the high-dimensional residual stream, with features tiling the manifold. QK rotation (attention) performs boundary detection by rotating and aligning one manifold to another (line width representation). Multiple boundary heads cooperate with different offsets to estimate remaining character count at high resolution. Next word length and remaining character count are represented in nearly orthogonal subspaces, making "line break decision" linearly separable.

Gibbs phenomenon (ringing) creates a rippled manifold—an oscillatory residual pattern (overshoot–undershoot oscillation such as

Moire and

Aliasing) that emerges when projecting signals from high to low dimensions or approximating continuous values in finite dimensions.