MoE Gating network

Creator
Creator
Seonglae Cho
Created
Created
2024 Mar 5 9:56
Editor
Edited
Edited
2024 Nov 22 20:57
Refs
Refs
MoEs are based on a gate network (or router) to determine which tokens are sent to which experts.
 
 
 
 
 
 
 

Recommendations