BigBird attention

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2023 Oct 15 3:1
Editor
Edited
Edited
2024 Mar 27 3:48
Refs
Refs

Block sparse attention

notion image
 
 

Not a Full self-attention

notion image
 
 
 
 

RoBERTa

Pegasus-x

 

Documents

 
 

Recommendations