BigBird attention

Created
Created
2023 Oct 15 3:1
Editor
Creator
Creator
Seonglae Cho
Edited
Edited
2024 Nov 26 12:47
Refs
Refs

Block sparse attention

notion image
 
 

Not a Full self-attention

notion image
 
 
 
 

RoBERTa

Documents

 
 

Recommendations