# Distributed inference

<!-- https://learn-kernels.com/chapters/distributed-inference -->

A single GPU cannot hold a frontier model, and a single request cannot use a whole cluster efficiently. Everything in this chapter is about the seam between those two facts: how a model gets split across chips, how experts get routed to and dispatched across nodes, and how the two arithmetic phases of generation end up running on different hardware entirely.

## 4 sections

1.  [5.1Parallelism, collectives, and topology](https://learn-kernels.com/chapters/distributed-inference/parallelism-collectives-and-topology)
2.  [5.2Mixture-of-experts serving](https://learn-kernels.com/chapters/distributed-inference/mixture-of-experts-serving)
3.  [5.3Prefill and decode disaggregation](https://learn-kernels.com/chapters/distributed-inference/prefill-and-decode-disaggregation)
4.  [5.4Production systems and serving benchmarks](https://learn-kernels.com/chapters/distributed-inference/production-systems-and-benchmarks)