Seonglae ChoReasoning length (direction) is encoded as a linear direction within the model's residual stream. In particular, middle layers control reasoning length. Found approximately 4% of attention heads that induce short reasoning.
Perform linear projection to remove the "short reasoning direction" component from the output weights (Wh_o) of these heads. Only 0.2% of total parameters modified. Short reasoning accuracy improved by +6.39%, overall average +3.34%. Does not excessively increase reasoning length, but only improves quality of short answers. Much more effective than the simple technique of forcing longer outputs by appending "Wait" tokens.