NVFP4 and MXFP4 multiply the same sixteen codes. The difference is one byte of scale per block, and it is worth 1.6 dB on Gaussian data, 4.2 dB on activations and about 9% of perplexity.
A wonderful dive into how FP4 works
NVFP4 and MXFP4 multiply the same sixteen codes. The difference is one byte of scale per block, and it is worth 1.6 dB on Gaussian data, 4.2 dB on activations and about 9% of perplexity.