Relative Positional Encoding

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 Mar 2 5:58
Editor
Edited
Edited
2024 Aug 13 13:47
Refs
Refs
Instead of encoding the absolute distance in the queries and keys, relative position encoding adjusts scores based on the “distance” between the queries and keys.
Relative Positional Encodings
 
 
 
 
 
 
 

Recommendations