Mixture-of-Experts

Math Foundations of Transformers and MoE Layers