Local/Global Attention

Creator
Creator
Alan JoAlan Jo
Created
Created
2024 Feb 21 9:24
Editor
Editor
Alan JoAlan Jo
Edited
Edited
2024 Mar 28 9:55
Refs
Refs

Global attention

naive elementwise multiplied soft attention for each hidden layer output
 

Local Attention

Local window hard attention + global soft attention
 
 
 

2015 10000 citations

Effective Approaches to Attention-based Neural Machine Translation
 
 

Recommendations