EcoCompute.
"Quantization doesn't always save energy. See for yourself."
Every quantization tool tells you how to quantize. We tell you whether you should.
From the preprint “Weight-Only Quantization Does Not Always Save Energy” — direct NVML energy measurements across four NVIDIA GPU architectures.
Inference energy per 1M tokens
Crossover curve · the bigger the model, the more quantization saves
Solid lines connect measured points only (direct NVML). INT8 was measured on A800 and RTX 4090 only, and sizes outside each GPU's measured range aren't drawn here. For fitted extrapolation beyond the measured range, use the Your model tab.
Latest run · RTX 4090 (Ada) · measured in our own open container
NF4 crosses over between 3B and 7B: +36% energy penalty at 0.5B → −29% saving at 7B. INT8 (bitsandbytes LLM.int8()) never saves energy here (+47% … +256%): it does lower instantaneous power, but decode throughput collapses to 9–19 tok/s (FP16: 39–62 tok/s), so every token ends up costing more.
All raw energy.json reports, the aggregated CSV, environment metadata and both figures are archived
under DOI 10.5281/zenodo.21528103,
CC BY 4.0 — a single-platform deep dive that sits alongside the main
v1.1.0 dataset, which remains
the reference for every other GPU here. Honest scope: n = 1 per configuration
(no std/CV yet, unlike v1.1.0's n = 2 / CV < 2%) and NVML measures GPU-package power,
not whole-system wall power — a supplementary case study, not a certified benchmark. Reproduce it on your own card
from the Run it yourself tab.
About EcoCompute
Evidence over hype — does quantization actually lower your LLM inference energy, or raise it?
EcoCompute answers a single question: does quantization actually lower the energy cost of your LLM inference — or raise it? Most quantization tooling tells you how to quantize; EcoCompute tells you whether you should.
The finding
Weight-only quantization (NF4/INT8) reduces memory footprint, but it does not always reduce energy. On small models the dequantization overhead can outweigh the memory-bandwidth savings, so quantization increases energy; on larger models it starts to save. The exact "crossover" point depends on both model size and GPU architecture.
How we measure
Every number on the "Tested models" tab comes from direct NVML power sampling during real inference (10 Hz, 256 tokens/run, 10 iterations) across four NVIDIA architectures (Turing, Ada, Ampere, Blackwell) — not TDP estimates. Where we model rather than measure (latency/throughput via a roofline model, or sizes outside the measured range), we label it explicitly. We never present estimates as measurements.
Our own measurement container — and the newest run
The measurement is open, not just the numbers. We develop the EcoCompute energy MLCube, an MLCube-compatible container that runs the same direct NVML power sampling on your GPU and writes an energy.json with the exact fields these charts read (see the Run it yourself tab). The newest column on this site — RTX 4090 (Ada), July 2026 — was produced with it: all 15 configurations (five models, 0.5B–7B × FP16/NF4/INT8) are real hardware measurements, and they extend our Ada coverage to INT8 and 7B for the first time. The complete run — raw per-configuration reports, aggregated CSV, environment metadata and figures — is archived on Zenodo under 10.5281/zenodo.21528103 (CC BY 4.0). It is a supplementary deep dive alongside the main v1.1.0 dataset, not a replacement for it and not a certified benchmark: n = 1 per configuration, and NVML reports GPU-package power rather than whole-system draw.
The research
Based on the preprint "Weight-Only Quantization Does Not Always Save Energy: An Empirical Study of LLM Inference Across NVIDIA GPU Platforms" by Hongping Zhang (Independent Researcher, Changsha, China; ORCID 0009-0000-2529-4613), openly available on SSRN. The measurement methodology — reproducible, direct NVML power sampling rather than TDP estimates — is the paper's core contribution, and the dataset is actively being expanded to cover more GPU architectures and model families.
Which artifact should I cite?
Three artifacts, three jobs — cite the one that matches your claim.
- The paper — the argument and the methodology: SSRN #6854700 (archived copy: 10.5281/zenodo.21066652).
- The main dataset — v1.1.0, current: 10.5281/zenodo.19647290 (CC BY 4.0). 360+ configurations, 0.5B–14B, four architectures (RTX 4090D, RTX 5090, A800, T4), FP16/NF4/INT8/FP8, n = 2 repeats with CV < 2%. This is the source of truth for the numbers on this page — cite it for anything except the RTX 4090 column.
- The RTX 4090 (Ada) deep dive — July 2026: 10.5281/zenodo.21528103 (CC BY 4.0). One card, 15/15 measured configurations from our own container, adding INT8 on Ada and 7B; n = 1, so it complements v1.1.0 rather than superseding it.
Older references still circulating: v1 (113 configurations, 10.5281/zenodo.18900289, March 2026) is superseded by v1.1.0 — mirrors or skill listings that still say “113+ configurations” are describing v1.
Build on it
- Measurement container: ecocompute-mlcube — the same code that produced the RTX 4090 dataset runs on your GPU
- Dataset & code: ecocompute-ai
- Also contributed as an energy-efficiency FAQ to the bitsandbytes docs.
For developers
The same engine that powers this page is available programmatically — build EcoCompute checks into your own tooling or agents.
- REST API —
GET /v1/estimate·GET|POST /v1/optimizeat ecocompute-estimator…workers.dev (CORS open) - OpenAPI — self-describing schema at /openapi.json for auto tool discovery
- MCP tool —
recommend_configfor Claude Desktop / Cursor
EcoCompute is an independent research project. Our goal is honesty over hype: we show what the measurements say — including when quantization is the wrong choice.