About EcoCompute

An independent, self-funded project answering one question: does quantization actually lower the energy cost of your LLM inference — or raise it?

Author

Acknowledgments

EcoCompute is a one-maintainer project, but it is not a one-person project. The following people made it better:

Funding

No external funding. No grant, no institutional budget, no vendor sponsorship, no cloud credits. GPU time — the rented RTX 4090 instances on which the deep-dive sessions were measured — and the domain and hosting were paid for by the author.

Competing interests

The author declares no competing interests. Specifically: no financial or employment relationship with NVIDIA, with any cloud or GPU-rental provider (including AutoDL, on whose rented instances the RTX 4090 sessions were measured), with Hugging Face, or with the maintainers of bitsandbytes, whose NF4 and LLM.int8() kernels are the subject of most of the measurements here. A short energy-efficiency FAQ was contributed to the bitsandbytes documentation on a voluntary, unpaid basis. Nothing on this site is sponsored, and no advertising or affiliate links are carried.

The one bias worth naming, because it cannot be declared away: a project whose headline finding is "quantization does not always save energy" has an incentive to keep finding that. The defence is that the raw reports are published, the container is open, and a run that contradicts the curve is the contribution we ask for.

How we measure

Every number on the Tested models tab comes from direct NVML power sampling during real inference — never a TDP or datasheet figure:

Quantity	Value
Power	nvmlDeviceGetPowerUsage polled at 10 Hz, integrated over the decode loop — GPU-package power, not wall-plug
Workload	Single-stream decode, batch size 1, 256 generated tokens per run, 10 iterations after 2 warmups, greedy
Baseline	Each model's own FP16 run in the same session; every ΔE% is relative to that
Hardware	NVIDIA RTX 4090 and RTX 4090D (Ada), RTX 5090 (Blackwell), A800-SXM4 (Ampere), Tesla T4 (Turing)
Software	Pinned per session and published: see the environment card in the evidence layer

Where a value is modelled rather than measured it is labelled in the UI and in the API response — measured, interpolated, extrapolated, estimated. Estimates are never presented as measurements. The full protocol, the fits and the known limits are on the methodology page.

What these numbers are not

• Not wall-plug energy: CPU, DRAM, PSU losses, cooling, PUE and CO₂e are all outside the measurement.
• Not a certified benchmark: the reports are MLCommons-style, not MLPerf results, and no LoadGen is involved.
• Not a statement about quantization in general: they are statements about bitsandbytes NF4 / LLM.int8() on the cards listed. A different kernel or serving stack can reverse the sign.
• Not all replicated: the main dataset is n = 2 (CV < 2%); the RTX 4090 deep dive is n = 1 per configuration, except five INT8 configurations repeated three times (CV of ΔE% 0.6–3.9%). The site draws single-trial points hollow.

Related work · and what makes this different

EcoCompute is not a competitor to the established energy benchmarks — it answers a narrower question they do not: for one model on one GPU, does switching FP16 → NF4/INT8 raise or lower energy per token? That needs a controlled A/B of precisions on identical hardware, which is why we sample GPU-package power directly instead of ranking systems or estimating carbon.

Project	What it measures	How EcoCompute differs
MLPerf Power	Audited whole-system wall power of submitted MLPerf Inference/Training runs, under LoadGen timing rules, with certified results	We report GPU-package power only and run our own warmup/iteration loop, so our numbers are a supplementary energy methodology — explicitly certified_benchmark_result: false, never an MLPerf result
MLPerf Client v2.0	Client-machine performance (geomean TTFT and 2nd+ token rate) of the models vendors ship — all of them already quantized (Q4_0, int4, MLX q4, NVFP4/FP8). Its tokens/Joule path is wall power via a Yokogawa analyzer, SPEC PTDaemon and a second host	We measure the FP16 baseline alongside the quantized model in the same session, in software, on one machine — so the difference quantization made is a number rather than an inference. Different boundary (GPU package, not wall) and not a certified result: how the two map onto each other →
Green500	Ranks whole supercomputers by GFLOPS/watt on HPL-class numerical workloads	Different unit and scope: one accelerator, LLM decode phase, joules per generated token — and a comparison between precisions rather than a league table
CodeCarbon / CarbonTracker	Workload-agnostic trackers: sample RAPL/NVML while your job runs and convert to CO₂e with grid-intensity factors	Complementary, not overlapping: they instrument whatever you run, we fix the workload (256 tokens, batch 1, single stream) so two precisions are comparable — and we publish no CO₂e, because that needs a PUE and a grid model we do not have
ML CO₂ Impact and TDP calculators	Estimate energy from hardware TDP × runtime	No TDP anywhere in our pipeline: every published point is an integrated NVML power trace, and anything modelled is labelled interpolated/extrapolated

Trademark note: MLCommons, MLPerf and MLCube are trademarks of MLCommons Association, referenced here nominatively. The maintainer is an individual observer member of MLCommons; EcoCompute is independently developed and is neither an official MLCommons benchmark nor endorsed by MLCommons.

Data and code availability

Everything behind the figures is public and archived under CC BY 4.0 (data) or Apache-2.0 (code):

• Main dataset v1.1.0 — 10.5281/zenodo.19647290: 360+ configurations, 0.5B–14B, four architectures, FP16/NF4/INT8/FP8, n = 2, CV < 2%. Source of truth for everything except the RTX 4090 column.
• RTX 4090 (Ada) deep dive — 10.5281/zenodo.22037483: three sessions, raw per-configuration reports and logs, same-run perplexity, and the n = 3 INT8 repeats.
• Measurement container — ecocompute-mlcube, with a full transitive dependency lock, because NF4/INT8 kernels change between releases.
• Site, fits and API — quant-energy: build/measured.csv holds every anchor, curves.json the fitted parameters.

Which artifact to cite for which claim, with concept vs version DOIs and BibTeX: Artifacts & citation.

Corrections policy

Corrections are published, not quietly patched. When a number changes, the change and its cause are written down — the derived-column erratum in the dataset record, the recomputed RTX 4090 deltas, and the notes that state where two of our own sessions disagree by a factor of two. If you find an error, open an issue; a measurement that contradicts the published curve is the most useful thing anyone can send.

Contribute a measurement

The project is looking for runs on additional GPUs, cloud instances, model families and quantization backends (AWQ, GPTQ, TensorRT-LLM in particular, since everything here is bitsandbytes). Start from the container page: one command produces a schema-validated energy.json, and the replications page checks it in your browser and overlays it on the curve before you submit it.

For developers

The same engine that powers the site is available programmatically. The API is optional: the pages compute every number locally from the fitted curves, so nothing here depends on the API being reachable.

• REST API — GET /v1/estimate · GET|POST /v1/optimize at ecocompute-estimator…workers.dev (CORS open)
• OpenAPI — self-describing schema at /openapi.json
• MCP tool — recommend_config for Claude Desktop / Cursor