Citadel Machine Learning Engineer Interview Questions

Make a hard MoE router differentiable

Differentiable Routing for Mixture-of-Experts (MoE) Context You are working with an MoE layer that routes each token to k experts (often k ∈ {1, 2}). ...