What Is a Mixture of Experts (MoE) AI Model — and Why It Matters

AI companies increasingly describe their biggest models as “sparse” or built on Mixture of Experts — a design that lets a model hold enormous knowledge while spending only a fraction of its computing power on any single answer. Understanding MoE clears up a real mystery in the AI news cycle: how a company can release a model with hundreds of billions of parameters and still price access to it far below smaller-sounding rivals.

What Mixture of Experts Actually Is

A conventional, “dense” neural network uses every one of its parameters on every input it processes. If a model has 70 billion parameters, all 70 billion do some work on every word you type. A Mixture of Experts (MoE) model instead splits most of its layers into many smaller sub-networks called experts — anywhere from eight to several hundred of them — and adds a small “router” network that decides, token by token, which handful of experts should handle that particular piece of input. The rest sit idle for that step.

The idea predates today’s chatbots. Google researchers described a sparsely-gated Mixture-of-Experts layer as early as 2017, scaling a language model past 100 billion parameters with only a modest rise in computation. Google’s Switch Transformer extended the approach in 2021, and Mistral AI’s Mixtral models carried it into mainstream open-weight releases in 2023. MoE has since become close to the default architecture for frontier-scale models, because it solves a concrete problem: how to keep growing a model’s total knowledge without a matching rise in the cost of running it.

How the Routing Actually Works

Inside a transformer-based model — the architecture behind virtually every modern chatbot — the standard feed-forward layers are replaced with MoE blocks. Each block holds a pool of experts and a lightweight router. For every token the model processes, the router scores all the experts and activates only the top few (commonly two, sometimes up to eight); their outputs are combined and passed on to the next layer. Because different experts tend to specialize during training — some lean toward code, others toward a particular language or style of reasoning — the router effectively assembles a custom sub-model on the fly for each token.

This is why MoE models are described with two numbers instead of one: total parameters and active parameters. Z.ai’s GLM-5.2, for example, is built with roughly 750 billion total parameters spread across hundreds of experts, but only about 40 billion of them activate for any given token. DeepSeek-V3 follows the same pattern at a smaller scale — 671 billion parameters total, 37 billion active per token. In both cases, the model behaves, in terms of stored knowledge, like a much larger dense model, while costing about as much to run as one a fraction of the size.

Why It Matters

Training and running AI models is bottlenecked by compute, and compute is expensive. A dense model matching GLM-5.2’s total capacity would need to run all 750 billion parameters on every request, multiplying the GPU time and electricity behind each answer. MoE breaks that link between total capacity and running cost — a direct reason several recent open-weight models have been able to match Western frontier systems on benchmarks while charging a fraction of the price per token. It isn’t the only factor behind AI pricing, but it is one of the clearest architectural reasons a “bigger” model isn’t automatically more expensive to use.

In the News

Z.ai’s GLM-5.2 — covered here when it launched — is a clear recent example: a roughly 744-billion-parameter Mixture of Experts model, released open-weight, that ranks among the top global AI systems while pricing at about a sixth of GPT-5.5’s per-token cost. For background on the model family behind it, see our explainer on what GLM is.

FAQ

Is a Mixture of Experts model the same as running several separate AI models?
No. All the experts are part of one trained network sharing the same layers, embeddings, and training process. The router only decides which internal pathways handle each token — it isn’t switching between independent models.

Does MoE make a model smarter, or just cheaper to run?
Mainly cheaper and faster for a given level of capability. MoE lets a model store more total knowledge without a matching rise in inference cost, but the quality gains come from that extra capacity being usable at all, not from the routing mechanism itself.

Which AI models use Mixture of Experts today?
Most frontier-scale and efficiency-focused open-weight models do, including GLM-5.2, DeepSeek-V3, and Mistral’s Mixtral family, among others.

Can I run an MoE model on my own hardware?
Open-weight MoE models can be downloaded and run locally, but because the full parameter set still has to be loaded into memory even though only a fraction activates per token, they typically need more RAM or VRAM than a dense model with the same number of active parameters.

Sources: Noam Shazeer et al., “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer” (2017); NVIDIA’s Mixture of Experts glossary entry; DeepSeek-V3 Technical Report (arXiv:2412.19437); Z.ai / Hugging Face model documentation for GLM-5.2.