# CUTLASS, CuTe, and CUDA Tile · Programming models and profiling CUTLASS is NVIDIA’s own answer to the same problem, aimed squarely at one computation: it is “a collection of abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and related computations at all levels and scales within CUDA,” decomposing that work into reusable, modular components. Through CUTLASS 2.x, that decomposition mirrored the GPU’s own hierarchy (thread, warp, threadblock) directly. CUTLASS 3.0 broke that mirroring on purpose: Hopper’s warp-group-wide instructions do not correspond to any single warp or thread-level concept, so tying the library’s structure to one generation’s hardware layout kept breaking on the next generation. What replaced it is CuTe, described as “a collection of C++ CUDA template abstractions for defining and operating on hierarchically multidimensional layouts of threads and data.” A **CuTe layout** (A mapping from a logical, multidimensional coordinate to a linear memory offset, expressed as a shape and a stride so that CuTe can compose and manipulate it with ordinary function operations.) is, at bottom, a function from integers to integers, and CuTe builds a full **layout algebra** (Operations, including functional composition, product (build a larger layout by repeating a smaller one), and divide (partition one layout according to another), for combining and manipulating CuTe layouts as if they were ordinary functions.) on top of that idea: composition, product, and divide let a kernel author build a thread-to-data mapping for an entire GEMM tile out of a handful of primitive layouts, instead of hand-writing a new iterator type for every architecture-specific access pattern. CUTLASS 3.0 replaced most of its 2.x-era named iterator types with this one vocabulary type, on the argument that a mapping expressed as an algebra can be checked at compile time: “if the code compiles, it’s probably correct.” The shape:stride notation reads mechanically. In the layout algebra documentation’s worked example B = (4,3):(3,1), the shape (4,3) says coordinates run over a 4 by 3 grid, and the stride (3,1) says a step in the first coordinate advances the underlying index by 3 while a step in the second advances it by 1: the doc’s own table evaluates B(1,0) to 3 and B(0,1) to 1. Because a layout is just a function from integers to integers, the same function can often be written more than one way; the documentation notes that a column-major layout like (\_2,\_4):(\_1,\_2) “acts identically to” \_8:\_1 for 1-D coordinates, and CuTe’s `coalesce` operation is the simplifier that finds the smaller spelling without changing the function: > Figure. coalesce, from the CuTe layout algebra documentation The algebra’s central operation is functional composition, R := A ∘ B with R(c) = A(B(c)), and the documentation calls it “the core of CuTe,” used “in just about every higher-level operation.” Its worked example composes A = (6,2):(8,2) with B = (4,3):(3,1), evaluates all twelve outputs by hand, and lands on the observation the whole library rests on: the result is itself a layout, ((2,2),3):((24,2),8), and it is compatible with B, meaning every coordinate of B is also a valid coordinate of the result. Selecting every third element of one layout through another never leaves the algebra, which is what lets the product and divide operations, and ultimately a whole thread-to-data partition for a GEMM tile, be built out of compositions and checked at compile time. > Figure. The producer-consumer pipeline. CUTLASS's asynchronous pipeline. Producer threads load tiles into a circular list of shared-memory stages while consumer threads compute on tiles one slot behind, the two sides synchronized through acquire, commit, wait, and release barrier operations. In the highlighted slot the load of tile k+1 runs while tile k is being computed: the software pipelining CUTLASS calls critical to hiding the latency of global memory loads.Illustrative numbers CUDA Tile IR, introduced in CUDA 13.1, takes the tile idea a level higher than either CUTLASS or Triton do on their own. Where CUTLASS and CuTe still hand a C++ (or now Python, via CuTe DSL) programmer explicit control over the thread-to-data layout, a **tile block** (The basic unit of execution in a Tile IR program: a single logical thread that computes over a whole multidimensional tile of data, with the mapping onto actual hardware threads left entirely to the compiler.) in Tile IR is written with no thread-to-data mapping at all; the compiler decides how a tile block’s work lands on real SM threads, the memory hierarchy, and tensor cores. Tile IR is positioned as a compilation target beneath higher-level DSLs, not a replacement for CUTLASS or Triton themselves, which is exactly the role it plays for Triton’s Tile IR backend described in the previous section. The programming model documentation shows what that looks like as actual code. A Tile IR program is a module of tile kernels declared with `entry`, and every value in a kernel is a tensor whose rank, shape, and element type are statically known; rank-0 tensors are scalars, and global memory is only ever reached through tensors built from pointer parameters. Its first worked example, a 128-element vector addition, spends most of its lines constructing a tile of 128 pointers from a scalar base pointer: an `iota` builds the offset vector 0 through 127, a `reshape` and `broadcast` replicate the base pointer across the tile, and an `offset` adds the two. The arithmetic itself is then three statements: > Figure. Tile IR vector addition, from the Tile IR programming model documentation The documentation’s summary of that kernel is the model in one sentence: “this code is written from a single thread of control, but its level of parallelism will be determined by the compiler.” Scaling past one tile reuses CUDA’s launch shape rather than replacing it: tile blocks group into a 1-d, 2-d, or 3-d tile grid, the grid size set at launch determines how many tile blocks run, and each block queries its position with `get_tile_block_id` and the grid’s dimensions with `get_num_tile_blocks`, the role `blockIdx` plays in chapter 1’s kernels, one level of hierarchy up. Source GEMM hierarchy and correctness-by-default framing from [CUTLASS 3.0 Design](https://docs.nvidia.com/cutlass/latest/media/docs/cpp/cutlass_3x_design.html). Layout algebra from [CuTe Layout Algebra](https://docs.nvidia.com/cutlass/latest/media/docs/cpp/cute/02_layout_algebra.html). Producer and consumer roles, stages, and barrier operations from the [CUTLASS pipeline documentation](https://docs.nvidia.com/cutlass/latest/media/docs/cpp/pipeline.html). Tile IR and tile blocks from the [Tile IR introduction](https://docs.nvidia.com/cuda/tile-ir/latest/sections/introduction.html) and [programming model](https://docs.nvidia.com/cuda/tile-ir/latest/sections/prog_model.html) docs. More in [further reading](https://learn-kernels.com/chapters/reading#cutlass-cute-tile).