# Learn Kernels > An interactive book on GPU kernels and inference performance. Every page is available as Markdown by appending `.md` to its URL. The whole book as one document is at https://learn-kernels.com/llms-full.txt. Figures are interactive. In the Markdown they appear as a titled placeholder with a link, because what they teach is the response to input, which does not survive being written down. ## 0. Introduction Why this work exists, and how to read it. What GPU performance engineering actually is: the gap between running a computation and running it at the speed the hardware allows. Covers why that gap exists, how this book is ordered, and how to read it depending on what you already know. - [Introduction](https://learn-kernels.com/chapters/introduction): chapter overview - [0.1 Why kernels matter](https://learn-kernels.com/chapters/introduction/why-kernels-matter): Section 0.1. - [0.2 How this book is organized](https://learn-kernels.com/chapters/introduction/how-this-book-is-organized): Section 0.2. - [0.3 How to read it](https://learn-kernels.com/chapters/introduction/how-to-read-it): Section 0.3. ## 1. Foundations How a kernel runs, and what limits it. The GPU execution model, from a kernel launch down to a single warp, and the compute-bound versus memory-bound boundary that determines how fast any kernel can run. Ends with the KV cache as the first place these fundamentals show up in inference. - [Foundations](https://learn-kernels.com/chapters/foundations): chapter overview - [1.1 Threads, blocks, and warps](https://learn-kernels.com/chapters/foundations/threads-blocks-and-warps): Section 1.1. - [1.2 Compute-bound and memory-bound](https://learn-kernels.com/chapters/foundations/compute-bound-and-memory-bound): Section 1.2. - [1.3 The KV cache](https://learn-kernels.com/chapters/foundations/the-kv-cache): Section 1.3. - [1.4 Compilation and machine code](https://learn-kernels.com/chapters/foundations/compilation-and-machine-code): Section 1.4. ## 2. Kernel optimization From naive matmul to fused, low-precision kernels. The foundational exercises everyone works through first, then matrix multiplication from a naive kernel to warp-level tiling, tensor cores and FP8 trading precision for throughput, and FlashAttention fusing attention into one IO-aware kernel. - [Kernel optimization](https://learn-kernels.com/chapters/kernel-optimization): chapter overview - [2.1 Foundational kernel exercises](https://learn-kernels.com/chapters/kernel-optimization/foundational-kernel-exercises): Section 2.1. - [2.2 Matrix multiplication](https://learn-kernels.com/chapters/kernel-optimization/matrix-multiplication): Section 2.2. - [2.3 Tensor cores and low precision](https://learn-kernels.com/chapters/kernel-optimization/tensor-cores-and-low-precision): Section 2.3. - [2.4 Attention](https://learn-kernels.com/chapters/kernel-optimization/attention): Section 2.4. ## 3. Programming models and profiling Blocked programs, tile layouts, and how to verify both. How Triton, the CUTLASS family, and their counterparts on AMD, TPU, and Trainium move kernel authoring from per-thread code to tile-level programs, and how Nsight Compute and Compute Sanitizer verify the result is fast and correct. - [Programming models and profiling](https://learn-kernels.com/chapters/programming-models): chapter overview - [3.1 Triton](https://learn-kernels.com/chapters/programming-models/triton): Section 3.1. - [3.2 CUTLASS, CuTe, and CUDA Tile](https://learn-kernels.com/chapters/programming-models/cutlass-cute-and-cuda-tile): Section 3.2. - [3.3 Other hardware stacks](https://learn-kernels.com/chapters/programming-models/other-hardware-stacks): Section 3.3. - [3.4 Profiling, benchmarking, and correctness](https://learn-kernels.com/chapters/programming-models/profiling-and-correctness): Section 3.4. ## 4. Inference engines Where throughput is actually won. How production engines batch, schedule, compress, and speed up decoding across many concurrent requests: continuous batching and paged KV memory, KV cache systems, quantization, speculative decoding, structured output, and long context. - [Inference engines](https://learn-kernels.com/chapters/inference-engines): chapter overview - [4.1 Scheduling and continuous batching](https://learn-kernels.com/chapters/inference-engines/scheduling-and-continuous-batching): Section 4.1. - [4.2 KV cache systems](https://learn-kernels.com/chapters/inference-engines/kv-cache-systems): Section 4.2. - [4.3 Quantization](https://learn-kernels.com/chapters/inference-engines/quantization): Section 4.3. - [4.4 Speculative decoding](https://learn-kernels.com/chapters/inference-engines/speculative-decoding): Section 4.4. - [4.5 Structured decoding and fairness](https://learn-kernels.com/chapters/inference-engines/structured-decoding-and-fairness): Section 4.5. - [4.6 Long context and multimodal inference](https://learn-kernels.com/chapters/inference-engines/long-context-and-multimodal): Section 4.6. ## 5. Distributed inference When one machine is not enough. Splitting a model across devices over collective communication, serving mixture-of-experts models, separating prefill from decode, and the production systems and benchmarks that measure the result. - [Distributed inference](https://learn-kernels.com/chapters/distributed-inference): chapter overview - [5.1 Parallelism, collectives, and topology](https://learn-kernels.com/chapters/distributed-inference/parallelism-collectives-and-topology): Section 5.1. - [5.2 Mixture-of-experts serving](https://learn-kernels.com/chapters/distributed-inference/mixture-of-experts-serving): Section 5.2. - [5.3 Prefill and decode disaggregation](https://learn-kernels.com/chapters/distributed-inference/prefill-and-decode-disaggregation): Section 5.3. - [5.4 Production systems and serving benchmarks](https://learn-kernels.com/chapters/distributed-inference/production-systems-and-benchmarks): Section 5.4. ## 6. Current hardware What the chips being deployed now provide. The architectures inference runs on today, read from their own specifications and tuning guides rather than from vendor peak numbers. - [Current hardware](https://learn-kernels.com/chapters/hardware): chapter overview - [6.1 NVIDIA Blackwell](https://learn-kernels.com/chapters/hardware/nvidia-blackwell): Section 6.1. - [6.2 AMD, TPU, and Trainium](https://learn-kernels.com/chapters/hardware/amd-tpu-and-trainium): Section 6.2. ## 7. Frontier Where the evidence is still settling. AI-generated kernels and their benchmarks, plus the hardware and techniques worth tracking before they have reproducible measurements behind them. - [Frontier](https://learn-kernels.com/chapters/frontier): chapter overview - [7.1 AI-generated kernels](https://learn-kernels.com/chapters/frontier/ai-generated-kernels): Section 7.1. - [7.2 Watchlist](https://learn-kernels.com/chapters/frontier/watchlist): Section 7.2. ## Glossary Every term, defined once. A searchable reference for the vocabulary of the book. Entries live in content/glossary.json. - [Glossary](https://learn-kernels.com/chapters/glossary): chapter overview ## Further reading The papers, guides, and repositories worth your time. Primary sources on GPU performance engineering, grouped from a single inference request down to a single kernel and back out to distributed serving. Entries live in content/reading.json and the groups in components/book/reading-list.tsx. - [Further reading](https://learn-kernels.com/chapters/reading): chapter overview - [B.1 Start here](https://learn-kernels.com/chapters/reading#start-here): Section B.1. - [B.2 GPU fundamentals](https://learn-kernels.com/chapters/reading#programming-model): Section B.2. - [B.3 Kernel optimization](https://learn-kernels.com/chapters/reading#foundational-kernel-exercises): Section B.3. - [B.4 Programming models and profiling](https://learn-kernels.com/chapters/reading#triton): Section B.4. - [B.5 Inference engines](https://learn-kernels.com/chapters/reading#scheduling-batching): Section B.5. - [B.6 Distributed inference](https://learn-kernels.com/chapters/reading#parallelism-collectives): Section B.6. - [B.7 Current hardware](https://learn-kernels.com/chapters/reading#hardware-nvidia): Section B.7. - [B.8 Frontier](https://learn-kernels.com/chapters/reading#ai-generated-kernels): Section B.8. ## Reference - [Glossary](https://learn-kernels.com/chapters/glossary): every term of art, defined once - [Further reading](https://learn-kernels.com/chapters/reading): the sources behind the book - [About](https://learn-kernels.com/about): what this site is - [Privacy](https://learn-kernels.com/privacy): what is collected, what Ask AI sends, and what is not ## For developers and agents Learn Kernels also publishes its chapter and section index as JSON, for programs that would rather not scrape HTML or Markdown. The API is versioned, rate-limited, keyless, and its errors are JSON: see the developer page for the details. - [Developer resources](https://learn-kernels.com/developers): quickstart, errors, versioning, rate limits - [OpenAPI spec](https://learn-kernels.com/openapi.json): the JSON API described machine-readably - [GET /api/v1/chapters](https://learn-kernels.com/api/v1/chapters): every chapter, as JSON - [GET /api/v1/chapters/{slug}](https://learn-kernels.com/api/v1/chapters/introduction): one chapter and its sections, as JSON - [MCP server](https://learn-kernels.com/api/mcp): the same chapter index as MCP tools, for MCP-native clients