Attention Value

Creator
Creator
Alan JoAlan Jo
Created
Created
2023 Aug 21 16:39
Editor
Editor
Alan JoAlan Jo
Edited
Edited
2024 Jun 29 7:31

Attention Output, 최종 값 (Context Vector)

Query, Keys, Value할 때 value와 다르고 거기에 attention weight 곱해서 나오는 게 attention value. 헷갈리니까 Attention output이라고 하자
Each word is also represented by a value which contains the information of that word. As a result, each context word is now represented by an attention-based attention value of all the words in the sentence.
 
 
 
 
 
 
 

 

Recommendations