In-context Vector

ICV

• Task Summary: Extract latent states from all layers of the final token by inputting each demo example into the model

• Feature Shifting: Add ICV to all tokens and all layers when processing a query input (adjust intensity with λ)

In-context Vectors: Making In Context Learning More Effective and...

Large language models (LLMs) demonstrate emergent in-context learning capabilities, where they adapt to new tasks based on example demonstrations. However, in-context learning has seen limited...

https://arxiv.org/abs/2311.06668

In-Context Learning at Representation Level via Unlabeled Texts

Large language models (LLMs) have exhibited impressive capability of In-Context Learning (ICL), where LLMs perform relatively complicated tasks beyond the pre-training objective by conditioning on...

https://openreview.net/forum?id=Af7CsWMUNI&noteId=R5kHXHQ4Pe