The Software Frontier | Lorenzo Bradanini

A wonderful dive into how FP4 works

NVFP4 and MXFP4 multiply the same sixteen codes. The difference is one byte of scale per block, and it is worth 1.6 dB on Gaussian data, 4.2 dB on activations and about 9% of perplexity.

READ THE LATEST

Recent posts

View all

Why Cached Tokens Cost 10% and Vanish in 5 Minutes

Every agent resends its whole context at every step, and every provider bills the repeat at a discount with a timer attached. The discount and the timer…

Sep 29 • Lorenzo Bradanini and Lorenzo Tettamanti

How Mojo Actually Compiles

Ask Mojo 1.1 for sm_90 and it quietly returns sm_90a, hinting that portability no longer lives in the binary. An H100 executable can carry kernels…

Sep 24 • Lorenzo Bradanini and Lorenzo Tettamanti

How NVIDIA's Data Center Business Actually Works in 2026

NVIDIA booked $89.0 billion of data center revenue in its latest quarter. The company now reports about $530 billion of commitments and maximum…

Sep 20 • Lorenzo Bradanini and Lorenzo Tettamanti

Unleashing OpenCL’s secrets

A technical account of the standard, its implementations, and where it actually stands in 2026.

Sep 10 • Lorenzo Bradanini and Lorenzo Tettamanti

Inside Google’s TPU: How it works, and what it costs

We ran it against five TPU generations, from v4 to Ironwood, then followed the money: a rate card that prices bandwidth, an anchor tenant paying 13% of…

5 hrs ago • Lorenzo Bradanini and Lorenzo Tettamanti

Subscribe to get deep engineering insights you won’t find elsewhere!

Where abstraction ends. Essays on GPU execution, kernel internals, and distributed systems at scale.

About Archive Recommendations Sitemap