MoEs are based on a gate network (or router) to determine which tokens are sent to which experts.
MoE Gating network
Creator
Creator
Seonglae ChoCreated
Created
2024 Mar 5 9:56Editor
Editor
Seonglae ChoEdited
Edited
2024 Nov 22 20:57Refs
Refs