Seonglae ChoReasoning length (direction) is encoded as a linear direction within the model's residual stream. In particular, middle layers control reasoning length. Found approximately 4% of attention heads that induce short reasoning.
Perform linear projection to remove the "short reasoning direction" component from the output weights (Wh_o) of these heads. Only 0.2% of total parameters modified. Short reasoning accuracy improved by +6.39%, overall average +3.34%. Does not excessively increase reasoning length, but only improves quality of short answers. Much more effective than the simple technique of forcing longer outputs by appending "Wait" tokens.
EMNLP 2025
ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models
ThinkEdit: A weight-editing approach to extend reasoning in LLMs, improving performance on math benchmarks.
https://lilywenglab.github.io/ThinkEdit/