GLU function

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 Mar 30 17:50
Editor
Edited
Edited
2024 Mar 31 4:59
Refs
Refs

Gated Linear Unit

Gated mechanism usually means point-wise operation between sigmoid function to control flow (the function should not be a sigmoid but usually sigmoid is used)
Gating Mechanism
과 달리 weight는 없다
GLU functions
 
 
 
 
 
 

Recommendations