What is MXFP8 (E4M3)?
FP8 E4M3 (4 exponent, 3 significand bits) was proposed by NVIDIA, Arm, and Intel in September 2022 as the precision-optimized half of a dual-format FP8 scheme: E4M3 for weights and activations, E5M2 for gradients. The OCP MX consortium absorbed both encodings into the MXFP8 standard in 2023, wrapping them with 32-element block scaling. NVIDIA Hopper SM architecture (2022) was the first GPU with FP8 E4M3 silicon support, as a pre-standard per-tensor format without block scaling; MXFP8 with block scaling arrived with Blackwell.
What are blockwise quantization/micro-scaling float formats?
Standard formats like FP16 encode each element independently, with one exponent and significand per value. Micro-scaling formats like OCP MXFP4 trade that independence for compression: every 32 consecutive elements share a single scale factor (stored as an E8M0 power-of-two), and each element stores only its relative magnitude within the block in a low-precision E4M3 value.
The banding in the image above is the block structure made visible. A block containing both a very bright and a very dark pixel must scale to fit the bright one, collapsing the darker values into only a handful of distinct levels. FP4 has just 8 non-negative representable values (0, 0.5, 1, 1.5, 2, 3, 4, 6 × scale), so FP4 blocks "posterize" to at most 8 colors. FP6 has up to 28 non-negative values and FP8 up to 240, so degradation at those precisions is subtler.
These quantized formats are used in LLM inference to reduce demand on memory bandwidth, especially during decode, and to take advantage of higher arithmetic bandwidth, especially during prefill. They are generally destined for use in the Tensor Cores, where the vast majority of that bandwidth lies in contemporary GPUs.
Explore how individual floats in these formats are encoded on the Quant Formats page. The image-as-tensor visualization technique is inspired by quant-jaunt.
