Auxiliary-Loss-Free Load Balancing

Loss-Free Balancing selects experts according to a “biased gating score” in each training step and updates this expert-wise bias after each training step