Seonglae ChoInstead of manipulating activations, this lightweight technique proposes injecting vectors only once into the KV cache (key/value tensors of each layer) to guide the model's reasoning (Chain-of-Thought) tendency. From contrastive pair prompts (positive: includes CoT, negative: answer only), K/V steering vectors are generated per layer using CAA.
In terms of the coefficient, while the original paper used manual tuning, it was meaningful that it showed stability and good effects on reasoning benchmarks even in larger models.