The first independent replication: NF4 saved 16% on a laptop GPU

Every number on this site so far was measured by one person on rented cards. On 23 August 2026 that stopped being true: @gkgoing ran the container on a laptop — an RTX 3050 Ti Laptop GPU under Windows 10 — and submitted the report. Thank you. The raw file is in the repository verbatim and the point is now drawn on the fitted chart and in the gallery.

It disagrees with us, in the most useful direction possible.

What the run says

field	value
GPU	NVIDIA GeForce RTX 3050 Ti Laptop GPU · 39.8 W average package power
Model / precision	TinyLlama-1.1B-Chat-v1.0 · NF4 (bitsandbytes)
Energy	1212.9 mJ/token vs an FP16 baseline of 1448.2 mJ/token
ΔE vs FP16	−16.2%
Perplexity	10.1016 vs FP16 9.6201 → +5.005%
Throughput	32.8 tok/s
Stack	Windows 10 · driver 576.52 · Python 3.13 · torch 2.6.0+cu124 · transformers 4.57.6 · bitsandbytes 0.43.3
Protocol	direct NVML @ 10 Hz · batch 1 · 256 tokens · 10 decode iterations · 2 warmup · n = 1

Nothing we own behaves like this at 1.1B

TinyLlama-1.1B in NF4 is the single configuration we have measured on the most cards. On every one of them, quantizing it costs energy:

Card	ΔE vs FP16 at 1.1B, NF4	NF4 energy	n
RTX 4090D (Ada)	+33.4%	2134 mJ/token	2
RTX 5090 (Blackwell)	+26.5%	2098 mJ/token	2
RTX 4090 (Ada, July)	+16.3%	1884 mJ/token	1
RTX 4090 (Ada, 19 Aug session)	+5.5%	1696 mJ/token	1
T4 (Turing)	+4.6%	4445 mJ/token	2
RTX 3050 Ti Laptop (Ampere) — contributed	−16.2%	1213 mJ/token	1

Read the third column again: the laptop is not only the one card where NF4 pays off at this size, it is also the lowest absolute energy per token in the set — below a 4090, at a third of the power and a fraction of the price. That is one run on one machine, so treat it as a lead, not a law. But it is the kind of lead that would never have come out of our own hardware budget.

Does it break the published curve? No — the curve had nothing to say there

The submission page told the contributor his point was “outside the band by 9.9 pts”. That was our bug, and this run exposed it. The fitted Ampere NF4 curve rests on three A800 anchors at 7B, 9B and 14B. At 1.1B it is extrapolated more than six-fold below anything it ever saw, so its honest 95% band there is roughly ±30 points — wide enough to swallow almost any result. The page was comparing against the in-sample residual spread (±5.3) and calling an untested region a contradiction.

It now uses the same band as the rest of the site — the fit's own instability, widened by log-distance outside the measured range (how the band is built) — and when a point lands in a region that wide it says so, instead of claiming either a confirmation or a refutation:

What we are not doing with it

We are not folding it into the fit. Two reasons, both worth stating out loud:

• n = 1. Same standard we hold our own single trials to — drawn faded and labelled provisional.
• It would pool a 4 GB laptop card with an 80 GB datacenter card under the label “Ampere”. We already flag the one place we pool two cards (Ada: RTX 4090 + 4090D, which disagree by about 20 points at the same size). Adding a third, far more extreme pooling to hide an inconvenient point would be worse than leaving the point unfitted and visible.

So contributed points are drawn on the charts as violet diamonds and never enter curves.json or build/measured.csv. The curve does not move to meet them. If a second and third laptop-class run agree with this one, the honest response is a new architecture class fitted to laptop cards — not a quietly nudged Ampere curve.

The quality axis replicated to four decimal places. The energy axis did not replicate at all.

This is the part we did not expect to get for free. The container scores perplexity on a fixed vendored text after the power sampler stops. Our RTX 4090 measured TinyLlama-1.1B in NF4 at 10.1014 against an FP16 baseline of 9.6199. The laptop, on a different operating system, a different torch build and hardware two orders of magnitude apart in price, measured 10.1016 against 9.6201 — the same +5.005% penalty.

Two axes, measured in the same run, behaving in opposite ways:

• Quality is a property of the weights and the corpus. It travels. A perplexity delta measured on a laptop is a perplexity delta on an H100.
• Energy is a property of the hardware and the kernel path. It does not travel — not between cards, and, as we found in our own two sessions, not always between days on the same card.

Which means a paper claiming “4-bit quantization saves X% energy” without naming the card is not making a weaker claim than it should — it is making no claim at all. We had asserted that. Now there is evidence from someone else's hardware.

Why a laptop might flip the sign — a hypothesis, and how to kill it

Decode at batch 1 reads the whole weight matrix per token. NF4 cuts that traffic roughly four-fold and pays for it in dequantization work. On a 4090, at 1.1B, there is bandwidth to spare and the dequantization overhead dominates, so the model gets slower and the energy goes up. On a ~40 W laptop part, the same 1.1B model is comparatively bandwidth-starved, so the traffic saved should be worth more than the overhead costs.

That is a hypothesis, untested, and it is not the only candidate. This run carries the fourth distinct torch build in this record — 2.6.0+cu124 here, against 2.4.1+cu121 and 2.5.1+cu121 in our sessions and 2.13.0 pinned in the container image — and we already suspect the kernel path of causing a 100–140 point swing between two of our own sessions. Part of the −16.2% may be the stack rather than the card. What narrows it usefully: the contributor's transformers 4.57.6 and bitsandbytes 0.43.3 are the exact versions of our 19 August 4090 session, the +5.5% row above. Same quantization library, same inference library, opposite sign.

Both are cheap to separate, and neither needs our hardware:

• Run 0.5B, 1.5B and 3B NF4 on the same laptop. If the bandwidth story is right, the savings grow with size and the laptop's crossover sits below 1.1B, where every desktop card's crossover sits above 3B.
• Run INT8 on the same laptop. Our data says INT8 never saves energy on any card, at any size. If a laptop breaks that too, the claim needs rewriting, not defending.
• Run the same NF4 configuration on a 4090 with torch 2.6.0+cu124 — the only version that differs from our 19 August session. If our 4090 number moves toward the laptop's, the stack is doing part of the work.

Caveats we are not hiding behind

• n = 1, so there is no measured spread. Our own repeated runs put run-to-run CV at 0.6–3.9%, which is nowhere near 16 points — but that CV was measured on a datacenter card in a rack, not on a laptop with thermal and battery management we cannot see.
• GPU package power only, via NVML — no CPU, DRAM, PSU losses, PUE or CO₂e. On a laptop the package share of system power is smaller than on a desktop card, so this number is even further from a wall-plug measurement than ours.
• We moderate for schema, never for results. Nothing in this report was re-run, corrected or checked against a second machine by us. It is published as submitted, and @gkgoing can withdraw it at any time by saying so in the issue.

Be the second

One command, then drop the file on the replications page — it is parsed in your browser and nothing is sent until you send it yourself:

docker run --rm --gpus all -v "$PWD/out:/out" \
  ghcr.io/hongping-zh/ecocompute-mlcube:latest \
  --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --precision NF4 \
  --output_dir /out

Points that contradict the curve are the valuable ones, and they are published exactly as prominently as the ones that confirm it. This one already changed a chart, a band calculation and a claim about what quantization does on small hardware — on its first day.