Encoder Self-Attention

Self-attention enables the encoder to weigh the importance of each word and capture both short and long-range dependencies.