The 4-Bit Model That Beat Its BF16 Source Also Trained Longer

A compressed model can improve while its weights move to four bits. That does not mean reducing their precision made it better.

Multiverse Computing's August 25 Quantization-Aware Healing post reports a 60-billion-parameter model, cut down from GPT-OSS 120B, that matches or beats its BF16 predecessor on seven of nine benchmarks after conversion to MXFP4. The conversion step, however, includes another round of training against the original 120B model. The BF16 checkpoint it is compared with stopped one pass earlier.

For a serving team, the result still answers a useful question: can the low-precision artifact recover enough quality to become a deployment candidate? It leaves a second question open. How much of the gain came from training under quantization, and how much came from more teacher supervision? The blog's first-listed author, Antonio Tiene, has since acknowledged in a public reply that the headline table does not separate them. Keeping the two questions apart makes the result easier to use.

What the healing pass actually trains

The technical paper describes three modifications to the original model. A proprietary structural compression operator first reduces GPT-OSS 120B to 60B parameters. Training in BF16 then recovers some of the lost capability by distilling the original model's output distribution into the smaller one. Quantization finally casts the recovered student's weights to MXFP4, which adds a further accuracy gap on top of the compression loss.

Quantization-Aware Healing, or QAH, trains that quantized student against the original 120B model a second time. Fake quantizers, which simulate MXFP4 rounding, sit in the student's forward pass, so training encounters the effects of the intended low-precision representation. The objective is a KL divergence between the teacher's and student's predicted token distributions. The student sees no hard labels. For the 60B student, the paper reports a 32k-token sequence length, a global batch of 64 and 400 steps, with embeddings, layer norms and selected attention components frozen.

Because the loss compares teacher and student predictions over a shared output vocabulary, their internal architectures can differ. That is the engineering separation the method depends on. It lets QAH aim the compressed student at the original 120B model's behavior instead of at the recovered 60B checkpoint, which is itself a distilled approximation of that model.

The teacher's size and its numerical precision are separate facts. The blog's results table labels the teacher "120B teacher (MXFP4)", while its approach section calls the teacher full-size and full-precision. The paper resolves the inconsistency: the original GPT-OSS release already stores most of its weights in MXFP4, with the remaining tensors in BF16, and the authors write that the advantage comes from the teacher being uncompressed, not from a difference in precision. The comparison is therefore between an uncompressed, mostly 4-bit teacher and a compressed 4-bit student. It is not a uniformly 16-bit teacher against a uniformly 4-bit student.

Whether the original model actually makes a better teacher than the recovered checkpoint has not been measured directly. The paper's limitations section says the authors did not run QAH head-to-head against distillation from the recovered BF16 checkpoint at matched configuration. They call it the single most valuable experiment to add, and say their claim that a recovered teacher caps the student's accuracy should be read as well-motivated rather than demonstrated. That is one missing comparison. The headline table lacks another.

Compare training histories before comparing precision

On August 29, a reader commented under the blog that the headline table had no control: the BF16 model received one round of distillation and the MXFP4 model received that round plus a second one. On August 30, Tiene agreed. He acknowledged that the table could not distinguish the effect of training under quantization from that of training longer, and that the team had overstated what it showed about quantization itself.

The reply names the controls that would separate the effects. Continue the BF16 checkpoint with the same teacher, data and step count. Then take that continued BF16 checkpoint and quantize it to MXFP4 after training. Tiene called the second comparison the decision a deploying team actually faces: heal under quantization, or quantize afterward. Both arms were proposed for a possible revised version. Neither is reported as a completed experiment.

The paper itself is more careful than the blog on this point. Its results section says the finding does not establish that 4-bit representations are inherently superior, only that the deployable artifact is at least as capable as its BF16 source given the extra distillation pass. Tiene's reply said the team would add a training-history caveat to the blog text. In the blog version PSEEDR read on November 1, the body text still described the teacher as full-precision, and the acknowledgement remained in the comment thread rather than the article.

Tiene defended the narrower checkpoint comparison: the 4-bit result scores above the BF16 checkpoint it was quantized from, where direct post-training quantization usually scores below. A team comparing artifacts can use that. It does not establish that QAH was the best way to spend the training budget, because the alternative routes were not run.

The reported gains also vary by task. The long-context reasoning score rises from 35.3 to 42.7, and AIME 2025 rises from 70.7 to 76.3. MMLU-Pro slips from 74.0 to 73.8, and SciCode falls from 35.6 to 34.2. Every number is a single run with no seed variance or confidence intervals, and AIME 2025 has 30 problems. The paper reads the half-point LiveCodeBench advantage over the original teacher as a match within noise, not a win. A count of benchmarks won is a starting description, not a substitute for checking the workload that matters.

The released Hypernova-60B is a further distinct object. The paper says the public release received additional training beyond the pipeline it evaluates, and that its tables are the authors' own measurements of the pipeline rather than the released checkpoint's figures. A team that downloads Hypernova and runs its own benchmarks is evaluating a different artifact from the one in those tables. The published numbers should not be relabeled as a measurement of the downloadable model.

The healing job has its own memory bill

Lower serving precision does not remove the resources needed to produce the model. Distillation needs teacher outputs, student training state and the intermediate values used to compare the two distributions.

For long sequences, those intermediate values are the problem. A direct implementation forms a value for every batch position, sequence position and vocabulary entry. At 32k tokens and a vocabulary around 200,000 entries, the paper says that object becomes prohibitive. Reducing the student's parameter storage does nothing to it.

The paper's implementation precomputes teacher outputs offline and keeps the top 100 logits per position. It then evaluates the KL loss in sequence chunks instead of materializing the whole intermediate at once. Chunking cuts the peak working space for that operation. It does not make teacher generation, stored targets or the rest of training disappear.

This matters when deciding whether to reproduce the recipe. A compressed checkpoint alone is not enough. The described process also needs logit-level access to the uncompressed teacher, a training corpus, and a training system that fits the chosen context length. The paper's runs use eight nodes of H200 GPUs. Being able to serve the final model does not establish being able to train it.

The paper's sweeps also report two settings a team would otherwise have to rediscover. Unfreezing layer norms and embeddings during the quantization stage, especially at higher learning rates, produced checkpoints worse than the unhealed model. In the QAT sweep on the 120B student, the best FSDP2 configuration scored higher on GPQA Diamond than any tested DeepSpeed configuration. The best configurations differed in other training settings, and the authors leave the underlying cause unresolved. They treat the distributed backend as a setting to evaluate rather than an interchangeable choice.

The blog's claim that QAH is roughly seven times faster than quantization-aware training (QAT) also needs its conditions attached. On a smaller GPT-OSS 20B-to-9B configuration, built for an internal client deployment with details withheld, QAH reached its peak average in about 100 steps against about 700 for QAT on a cross-entropy loss. It is a step count under one configuration, not a measured wall-clock or cost figure, and QAH's steps are preceded by teacher-logit generation that the cross-entropy baseline does not require. The same figure shows QAT losing nearly 19 points by step 1,200 after its peak while QAH stayed within about two points of its own. That is a reported result from one configuration and single runs. The blog's statement that a sufficiently trained QAH checkpoint "can be served safely" generalizes from that run.

Measure the artifact in the serving system

The authors report roughly fourfold lower weight memory than the BF16 student. That is a narrower claim than fourfold lower total GPU memory or fourfold faster requests. The blog's statement that the model roughly halves compute per token relative to the 120B teacher is reasoned from the halved parameter count rather than measured, and its figure of about eight times less compute for model families shipped in BF16 is a projection for models the paper did not test. The serving runtime still has to support MXFP4 and accommodate the workload's other memory and compute demands.

An adoption test should therefore load the intended checkpoint in the intended engine, run the application's prompts at expected concurrency, and record task outcomes and resource use together. A model whose weights fit may still miss the required context length or throughput. A weight-memory reduction can also be worth having even if latency barely moves, because it may change which hardware configuration is feasible at all. These are engineering considerations drawn from the mechanism. The QAH study supplies no serving measurements.

The research covers one model family, one low-precision format and one data mixture, produced with a proprietary compression operator. The authors list all four limits themselves. The results justify investigating this recipe for that kind of constrained deployment. They do not establish equivalent gains for other compression methods or formats.

The next decision is concrete. Compare the low-precision artifact that can actually be served, keep its training history attached to the comparison, and run the missing alternatives if the choice of training recipe is what matters. A 4-bit checkpoint that beats its BF16 source is a useful outcome. Whether healing under quantization beats quantizing after the same amount of training is the question the authors have agreed is still open.

Reporting note: PSEEDR read the blog, its comment thread and the linked paper, arXiv 2608.20953 version 1 dated August 21, on November 1, 2026. The August 30 reply postdates the blog's August 25 publication. The analysis uses the versions available at reading time, not an archived August snapshot, and PSEEDR did not reproduce any training or serving result.