How numbers are stored
Later chapters will say things like “E4M3, with a 4-bit exponent and 3-bit mantissa,” and will treat the choice between number formats as one of the largest levers available. This section says what those words mean, because the lever only makes sense once you can see what is being given up.
A floating-point number is stored in three fields. One bit records the sign. A group of bits holds the exponent, which is the number’s scale, a power of two. The rest hold the mantissa, also called the significand or the fraction, which holds the significant digits. Scientific notation works the same way: in 6.02 times 10 to the 23rd, the exponent is 23 and the mantissa is 6.02.
The split between those two fields is the entire design decision, and it is a zero-sum one. Exponent bits buy dynamic range, the span between the largest and smallest magnitudes you can represent at all. Mantissa bits buy precision, how finely you can distinguish two nearby values. A format with too little range cannot hold the number and overflows to infinity or underflows to zero. A format with too little precision holds a number close to the one you wanted. Those are different failure modes, and which one hurts depends entirely on the workload.
IEEE 754, the standard nearly all of this follows, defines the 16-bit format as “1 sign bit, 5 exponent bits, and 10 fractional bits.” The consequences are stark when set beside 32-bit floating point. Half precision’s dynamic range, counting denormals, “is 40 powers of 2. For comparison, single precision dynamic range including denormals is 264 powers of 2.” Halving the storage does not cost half the range. It costs most of it.
Why fewer bits pays twice#
Given that cost, the pressure to use narrow formats anyway is worth explaining, and it comes back to the two budgets of the first section. Narrowing a number helps both of them at once, which is rare.
NVIDIA states both halves plainly. On bandwidth: “half-precision halves the number of bytes accessed, thus reducing the time spent in memory-limited layers.” On arithmetic: NVIDIA GPUs “offer up to 8x more half precision arithmetic throughput when compared to single-precision, thus speeding up math-limited layers.” One change moves the ceiling on both budgets, so it helps whichever one you happened to be against. There is a third effect too, the one that decides what you can serve at all: fewer bytes per parameter means “reduced memory usage of the neural network, allowing training and deployment of larger networks.”
This is why precision is not a detail in this book but a recurring lever. It is also why the formats that matter in practice are not simply IEEE’s. Once you accept that the exponent and mantissa split is a choice, you can make a different one: keep the exponent wide so the range survives and spend the loss on mantissa bits instead, or attach a shared scaling factor to a block of values so that a narrow format only ever has to represent numbers near a known magnitude. Those two ideas generate most of the format zoo in Tensor cores and low precision, which takes up TF32, BF16, FP8 and FP4 with the bit layouts and the hardware that consumes them. Quantization then covers the harder version of the question, where a model trained in one format is converted to a narrower one afterwards and has to be checked for what the conversion broke.
One further wrinkle explains a pattern you will see constantly in the later chapters: the precision a value is stored in and the precision it is computed in need not be the same. Rounding error accumulates, and a long chain of additions in a narrow format drifts in a way that a single multiplication does not. Hardware is built around that asymmetry. NVIDIA describes a Tensor Core as performing “D = A x B + C, where A, B, C, and D are matrices,” in which “A and B are half precision 4x4 matrices, whereas D and C can be either half or single precision 4x4 matrices.” Put plainly: “Tensor Core math can accumulate half precision products into either single or half precision outputs.” The multiply happens narrow, where the throughput and bandwidth savings are, and the running total is kept wide, where the error would otherwise compound. Whenever a later chapter says a kernel multiplies in FP8 and accumulates in FP32, this is the arrangement being described.
One last piece of vocabulary, because the later chapters switch between two kinds of number without announcing it. Integers have no exponent field at all: an 8-bit integer represents 256 evenly spaced values and nothing else, so using one to hold a weight requires deciding in advance what range those 256 values cover. That decision, and the scaling factors that carry it, is what makes integer quantization a different activity from simply picking a smaller float.