Attention Output

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2023 Aug 21 16:39
Editor
Edited
Edited
2026 Mar 8 15:29

Context Vector, Attention Value

Attention output instead of value to avoid confusion with the value in Query, Keys, Value
Each word is also represented by a value which contains the information of that word. As a result, each context word is now represented by an attention-based attention value of all the words in the sentence.
 
 
 
 
 
 
 

 

Recommendations