Short, evidence-based notes on the energy cost of LLM inference — grounded in direct NVML power measurements across NVIDIA GPUs. Honesty over hype. Positioning: this is the exploratory layer of the site — deeper write-ups, work-in-progress analyses and how-to guides. Results here are not counted in the coverage matrix or the seed counts; measurements that have been accepted into the grid live in the evidence layer, and the estimator and optimizer tools are separately labelled as model output.
The estimator now speaks MCP: one URL, four tools (estimate / crossover / coverage / search), no key. The design point is that every answer embeds its basis label, confidence, measurement scope and DOIs in the same text as the number — a model relaying it cannot strip the caveats. And "no data for your GPU" is a first-class answer, so an assistant wired to these tools learns to invite a measurement instead of inventing a number. The machine-readable version of this project's whole methodology.
A decision model named after the Jevons paradox advertises 193.6×; the community-measured median is 7×. Between the two stand four causes — baseline choice, task asymmetry, propagation echo, deployment physics — and one craft: choosing the measurement window where the invoice reads zero. "Outputs are free" is a pricing policy, not a physics result; every window has bills outside it, and some of them are fifty cents per successful attack. A commentary; no independent measurements of Jev.
We published an energy measurement and called it a probable overestimate in the same breath. This is the rerun, written as a protocol note: forced cooldown before every run, randomized order, --ignore-eos, a third token length to test the differencing assumption (R² ≥ 0.9966), and NVML's hardware energy counter — which reads 14.9% higher than integrating the same run's 100 Hz power trace. The headline moved 1.7 points; of the 50 minutes of GPU time, 45 were spent waiting for the card to cool down.
MLPerf Client v2.0 shipped an already-quantized suite, so we measured the FP16 baseline it cannot produce. Inside one llama.cpp build on one RTX 4090, GGUF Q4_0 costs 61.9% less decode energy per token than GGUF F16 for +5.60% perplexity — while drawing more power, 296–319 W against 273 W, so the saving is throughput and nothing else. On the same card bitsandbytes LLM.int8() costs 106% more. Three weight-only quantizations of a 7–8B model, spanning +106% to −62%: the bit width does not predict the sign. Re-run 2026-09-03 over 45 runs with cooldowns, randomized order and a hardware energy counter; the provisional −63.6% was an overestimate and the correction is documented in the note.
Someone else ran the container. On an RTX 3050 Ti Laptop GPU, TinyLlama-1.1B in NF4 saves 16.2% energy — where every card we own spends 4.6–33% more at that size, and where our own fitted curve was extrapolating six-fold beyond its anchors. The paired perplexity, meanwhile, reproduced our RTX 4090 measurement to four decimal places: quality travels between machines, energy does not.
Five INT8 configurations re-run three times each on the same RTX 4090: the coefficient of variation of the energy delta is 0.6–3.9% — 30–50× smaller than the 100–140 point gap between two of our own sessions, so that gap is not sampling. Also, absolute joules drifted 12–17% lower overnight on the same host while the FP16-normalised delta held.
Ten RTX 4090 configurations where the joules and the perplexity come from the same run. NF4 saves 39% energy at 7B but costs 12% perplexity; INT8 costs almost nothing in quality and never saves energy — a Δppl ≤ 1% filter rejects every NF4 row and admits nearly every INT8 one, the exact opposite of the energy ranking. Also: the same INT8 configuration measured 2.0–2.5× worse than in July, on the same card model and the same software stack.
在你自己的 GPU 上复现测量的逐步手册：三种运行方式（Docker 一行命令 / 租卡 bootstrap / 源码）、每个参数的作用、energy.json 各字段（含新增的困惑度质量块）、故障排查，以及每个数字的边界。A Chinese-language how-to for running the measurement container on your own GPU.
中文版站点导览：四个标签怎么用、每个数字的测量边界（GPU 芯片功耗、单流 batch 1、逐点标注的 n）、该引用哪个 DOI，以及四条必须记住的诚实声明。A Chinese-language guide to the site, its data boundaries and its citable artifacts.
15 of 15 configurations measured with direct NVML on one Ada card, using the open MLCube container anyone can run. NF4 goes from a +36% penalty at 0.5B to break-even at 3B and a −28% saving at 7B; INT8 cost more energy at all five sizes tested (n = 1 each, bitsandbytes LLM.int8()), because throughput collapses.
Weight-only NF4/INT8 quantization shrinks memory, but for small models it can raise inference energy by 25–55%. Here's what the measurements say, and how to find your crossover point.