Function vector

LLMs contain vector representations of tasks themselves, and inserting these vectors enables zero-shot task performance without examples. A small number of mid-layer attention heads aggregate task information to form function vectors (FV), with causal effects confirmed. Some FVs can be composed through vector addition to create new composite tasks.