About EcoCompute

An independent, self-funded project answering one question: does quantization actually lower the energy cost of your LLM inference — or raise it?

Author

Funding

No external funding. No grant, no institutional budget, no vendor sponsorship, no cloud credits. GPU time — the rented RTX 4090 instances on which the deep-dive sessions were measured — and the domain and hosting were paid for by the author.

Competing interests

The author declares no competing interests. Specifically: no financial or employment relationship with NVIDIA, with any cloud or GPU-rental provider (including AutoDL, on whose rented instances the RTX 4090 sessions were measured), with Hugging Face, or with the maintainers of bitsandbytes, whose NF4 and LLM.int8() kernels are the subject of most of the measurements here. A short energy-efficiency FAQ was contributed to the bitsandbytes documentation on a voluntary, unpaid basis. Nothing on this site is sponsored, and no advertising or affiliate links are carried.

The one bias worth naming, because it cannot be declared away: a project whose headline finding is "quantization does not always save energy" has an incentive to keep finding that. The defence is that the raw reports are published, the container is open, and a run that contradicts the curve is the contribution we ask for.

How we measure

Every number on the Tested models tab comes from direct NVML power sampling during real inference — never a TDP or datasheet figure:

Quantity	Value
Power	nvmlDeviceGetPowerUsage polled at 10 Hz, integrated over the decode loop — GPU-package power, not wall-plug
Workload	Single-stream decode, batch size 1, 256 generated tokens per run, 10 iterations after 2 warmups, greedy
Baseline	Each model's own FP16 run in the same session; every ΔE% is relative to that
Hardware	NVIDIA RTX 4090 and RTX 4090D (Ada), RTX 5090 (Blackwell), A800-SXM4 (Ampere), Tesla T4 (Turing)
Software	Pinned per session and published: see the environment card on the home page

Where a value is modelled rather than measured it is labelled in the UI and in the API response — measured, interpolated, extrapolated, estimated. Estimates are never presented as measurements. The full protocol, the fits and the known limits are on the methodology page.

What these numbers are not

• Not wall-plug energy: CPU, DRAM, PSU losses, cooling, PUE and CO₂e are all outside the measurement.
• Not a certified benchmark: the reports are MLCommons-style, not MLPerf results, and no LoadGen is involved.
• Not a statement about quantization in general: they are statements about bitsandbytes NF4 / LLM.int8() on the cards listed. A different kernel or serving stack can reverse the sign.
• Not all replicated: the main dataset is n = 2 (CV < 2%); the RTX 4090 deep dive is n = 1 per configuration, except five INT8 configurations repeated three times (CV of ΔE% 0.6–3.9%). The site draws single-trial points hollow.

Data and code availability

Everything behind the figures is public and archived under CC BY 4.0 (data) or Apache-2.0 (code):

• Main dataset v1.1.0 — 10.5281/zenodo.19647290: 360+ configurations, 0.5B–14B, four architectures, FP16/NF4/INT8/FP8, n = 2, CV < 2%. Source of truth for everything except the RTX 4090 column.
• RTX 4090 (Ada) deep dive — 10.5281/zenodo.22037483: three sessions, raw per-configuration reports and logs, same-run perplexity, and the n = 3 INT8 repeats.
• Measurement container — ecocompute-mlcube, with a full transitive dependency lock, because NF4/INT8 kernels change between releases.
• Site, fits and API — quant-energy: build/measured.csv holds every anchor, curves.json the fitted parameters.

Which artifact to cite for which claim, with concept vs version DOIs and BibTeX: Artifacts & citation.

Corrections policy

Corrections are published, not quietly patched. When a number changes, the change and its cause are written down — the derived-column erratum in the dataset record, the recomputed RTX 4090 deltas, and the notes that state where two of our own sessions disagree by a factor of two. If you find an error, open an issue; a measurement that contradicts the published curve is the most useful thing anyone can send.

Contribute a measurement

The project is looking for runs on additional GPUs, cloud instances, model families and quantization backends (AWQ, GPTQ, TensorRT-LLM in particular, since everything here is bitsandbytes). Start from the container page: one command produces a schema-validated energy.json, and the replications page checks it in your browser and overlays it on the curve before you submit it.

For developers

The same engine that powers the site is available programmatically. The API is optional: the pages compute every number locally from the fitted curves, so nothing here depends on the API being reachable.

• REST API — GET /v1/estimate · GET|POST /v1/optimize at ecocompute-estimator…workers.dev (CORS open)
• OpenAPI — self-describing schema at /openapi.json
• MCP tool — recommend_config for Claude Desktop / Cursor