From Formulas to a Real Model
Introduction
Project introduction
From Formulas to a Real Model
You have written the pieces, which are a zero point, group-wise int4 with protected columns, SmoothQuant's factors, GPTQ's column by column error feedback, NF4 levels and the size of a KV cache.
In the notebook you apply them to a small GPT-style model trained on Shakespeare and measure the loss on text it never saw after every change. Each method rounds the weights onto its levels and multiplies back, so the model runs in float32 but sees exactly the values a 4-bit kernel would.
What you will see
8 bits cost almost nothing. 4 bits with one scale per matrix send perplexity from 4.05 to 76 and the model writes gibberish. Smaller groups recover most of it, and the rest comes from a few input channels whose activations are 25 to 50 times larger than the others. Keeping them, or scaling them with AWQ, brings 4 bits within 0.02 of the float32 loss.
Large models grow those outlier channels on their own. This model is too small to, so they were planted with a rescaling that leaves every output unchanged. Without them, plain 4-bit rounding costs this model far less, so the large gaps you will see come from the planted channels, and the notebook says so where it matters. The same channels then hurt 8-bit activations and break 6-bit ones until SmoothQuant moves them into the weights, and planted outlier key channels decide how the KV cache should be quantized.
Perplexity, lower is better
32 bitsHow the notebook is organized
01
Formats and weights
Floats, zero points, scales per tensor, row and group, and the 4-bit twist.
02
Fixes
Protected channels, AWQ, GPTQ, SmoothQuant, NF4 and MXFP4.
03
Memory
The KV cache, and one table of bits, memory and loss.
Quantize a real language model
Run the whole investigation in the browser, about 75 minutes.