EcoCompute.

An open micro-benchmark for LLM quantization energy: a measurement protocol, a runnable container and an archived dataset. Independent open research project — building toward a reproducible inference-energy standard.

Measure whether LLM quantization actually saves energy — on your GPU.

Not TDP estimates: power is sampled directly from the GPU (NVML).
Enter your GPU and model size to get a direction; run the container to get your own measured number.

Estimate my model Run a real measurement See the full evidence

GPU-package power · not whole-system single-stream batch 1 · 256 tokens 26 / 56 cells measured Full scope and limitations →

Protocol v1.1 · normative 29 anchors · 5 GPU models · 4 arch. classes All counts defined · glossary NVML @ 10 Hz · GPU package MLCube-compatible Schema-validated Zenodo DOI Apache-2.0

Public replications →

What do you want to do? Pick one path

1

I just want the answer

Enter a GPU and a model size and get a directional verdict: does quantization save energy in your configuration, or cost more?

Open the estimator ↓ · Coverage matrix · Variable matrix

2

I want to verify it

No GPU needed: a free Colab T4 produces your own energy.json. With a GPU, run the container.

Verification steps ↓ · Container · Submit a replication

3

I want to cite or audit the research

Protocol, dataset DOIs, the limitations statement, the changelog and BibTeX — all checkable primary material.

Audit entry ↓ · Protocol · DOI / citation

① I just want the answer

Enter a GPU and a model size and get a directional verdict. The tool is right below: Tested models picks a card and model we actually measured; Your model takes your own model size and extrapolates the fit; Optimize searches precision × batch × context under your constraints.
Directional is not a production decision. See the limitations for the boundary of every number.

Inference energy per 1M tokens

Chart — inference energy per 1M tokens. Bars compare FP16, NF4 and INT8 on the selected GPU. Above zero means quantization costs more energy, below zero means it saves; the verdict line states the same result in words.
⚠ Conclusion boundary — read this before quoting the number

This is not a universal law for all GPUs, models or inference frameworks. Within the GPUs, software stacks, batch sizes, context lengths and measurement windows measured so far, NF4 shows a crossover that depends on model size and hardware — not “NF4 saves energy on big models, INT8 never does”.

NVIDIA GPUs only single-stream decode batch size = 1 256 tokens main dataset n = 2 GPU-package energy, not wall-plug outside the fitted range is not a measurement
Filled repeated runs — most trustworthy Hollow a single run — directional only Curve fitted / extrapolated — not a new measurement

Paste the verdict + citation into your tech spec, or share this exact config.

Crossover curve · in what we measured, the bigger the model, the more quantization saves

NF4 INT8 Your model Filled = n ≥ 2 repeated trials Hollow = n = 1 single trial Above zero = penalty (more energy) · below = savings
Chart — measured crossover curve. Energy change vs FP16 against model size. Marker shape encodes replication: filled = repeated runs, hollow = a single run. Shape and sign, not colour alone.
Filled repeated runs Hollow single run — directional Curve fitted / extrapolated

Not a universal law: NVIDIA GPUs · single-stream decode · batch 1 · 256 tokens · n = 2 · GPU-package energy, not wall-plug. Beyond the fitted range the curve is not a measurement. Full scope →

Solid lines connect measured points only (direct NVML). Marker fill encodes replication, not accuracy: filled dots are points repeated at least twice in the main dataset (v1.1.0, n = 2, CV < 2%); hollow dots on a faded line were measured once (n = 1) — the July 2026 RTX 4090 deep dive and one early RTX 4090D point — so they carry no measured spread and should be read as a single observation, not as a distribution. INT8 was measured on A800 and RTX 4090 only, and sizes outside each GPU's measured range aren't drawn here. For fitted extrapolation beyond the measured range, use the Your model tab.

② I want to verify it

The answer should not rest on us alone. This path gets you a comparable number on your own hardware — a result that contradicts ours is as welcome as one that confirms it.

1 · No GPU? Run the notebook in Colab

A free Colab T4 is enough — no GPU of your own, no Docker, nothing installed locally. The notebook runs the same NVML measurement as the container and writes the same energy.json, then prints an overlay link and a block of text you can paste into an issue.

Open the notebook in Colab → Read it on GitHub

It measures your chosen precision and its own FP16 baseline, in randomized order, waiting for the card to come back to idle before each arm — the two things we got wrong the first time and wrote up. Both arms have to fit: on a 16 GB T4 that means models up to about 3B, and the notebook refuses to start rather than failing with an out-of-memory error twenty minutes in. Budget 20–35 minutes; most of it is cooldown and the model download.

One pass is n=1 and energy is integrated from sampled power — on our RTX 4090 the GPU's hardware energy counter read on average 14.9% higher than integrating the same traces, so read these figures at that resolution. The last cell repeats both arms 5× if you want a repeatability figure. Same scope as everything else here: GPU-package power, not wall AC; not a certified benchmark result.

2 · Have a GPU? Run the container

One command, nothing to configure. Needs an NVIDIA GPU. With Docker it runs the prebuilt image; on a rented GPU instance that cannot nest containers (AutoDL, vast.ai …) the same command installs into a venv instead and runs the identical code. No GPU? It still runs and returns dataset-derived reference values, clearly flagged (never faked).

Quick start — pick a model and precision, one line:

curl -sSL https://quantenergy.tech/run.sh | bash -s -- --model qwen2.5-3b --precision nf4

Security model: quantenergy.tech/run.sh is a 20-line stub that downloads the real wrapper from pinned release run-v1 (immutable, checksummed), verifies its SHA-256, then executes it. DNS hijack of this site cannot change what runs — the stub always fetches from the pinned GitHub release URL. Known models: tinyllama-1.1b, qwen2.5-3b, qwen2.5-7b, phi-3-mini, yi-1.5-6b, mistral-7b. Use --model-id for any HuggingFace model. Add --dry-run to see the plan without executing. Audit the stub: run.sh. Audit the real wrapper: release asset.

Or verify first — download, checksum, then run (what the one-liner does internally):

curl -fsSLO https://github.com/hongping-zh/ecocompute-mlcube/releases/download/schema-1.3-r4/quickstart.sh curl -fsSLO https://github.com/hongping-zh/ecocompute-mlcube/releases/download/schema-1.3-r4/SHA256SUMS sha256sum -c SHA256SUMS && bash quickstart.sh

Download from an immutable release, verify against the published checksum, then run — what executes is a specific checksummed version, not whatever main holds today (bash quickstart.sh --dry-run prints the whole plan first). The container runs non-root, as your own uid. Measures NF4 and its own FP16 baseline on TinyLlama-1.1B, writes a schema-validated energy.json to ./ecocompute-out/ and prints an overlay link. The measurement itself takes a few minutes; the first run is dominated by downloads — the one-time image pull (or dependency install) plus a ~2 GB model download — and later runs reuse the caches. Our one timed end-to-end run took 57 minutes on a China-hosted rented RTX 4090 (mostly fetching torch/CUDA wheels; since fixed to use a domestic index, but not re-timed). No flags: the architecture comes from the NVML device name, so your card cannot be mislabelled. Other models/precisions and the plain docker run form: the container page.

Repo & full guide: github.com/hongping-zh/ecocompute-mlcube. This is a supplemental energy-methodology container (MLCube-compatible), not a certified benchmark.
The script ends with a link like quantenergy.tech/?tab=run&overlay=… — the point is encoded in the URL, so opening it restores your marker below with nothing uploaded.

3 · What you get — the same structure as this site

Example energy.json. The two fields the charts read are highlighted: vs_fp16_energy_pct (ΔE% vs FP16) and basis (measured / interpolated / extrapolated).


      
Shown here is the shipped example (examples/energy.measured.illustrative.json): its power/throughput numbers are synthetic only to illustrate the output shape. When you run on a real GPU, the same fields carry your actual NVML measurement (basis: "measured").

    


    

      
4 · Overlay your result on the crossover curve

      

        Drop your energy.json here, or click to choose a file
        
      

      
      
    

  


    

    

      
③ I want to cite or audit the research

      
This layer is for anyone citing, checking or recomputing. Every number carries its n, its window and its DOI.


      

        
Version matrix — what you are actually running

        

          
LayerCurrent versionDOI / releaseRole

          
Protocolv1.110.5281/zenodo.22958675Normative requirements

          
Report schemaecocompute-energy/1.3GitHub release schema-1.3-r4Machine validation

          
Containerschema-1.3-r4GitHub release · image digest in SHA256SUMSExecutable measurement

          
Main datasetv1.1.010.5281/zenodo.19647290Published measurements (360+ configurations)

          
SupplementRTX 4090 deep dive v310.5281/zenodo.22037483Deep re-test (Ada): repeatability, paired perplexity

        

        
The five layers version independently. “Running the project” today means the container release schema-1.3-r4, whose reports validate as ecocompute-energy/1.3 against Protocol v1.1 — and every published number traces to a dataset DOI above. Full version-by-version correspondence: spec §8.

      


      

        
Protocol & data

        
      


      

        
Limitations — read before citing

        
      


      

        
Citation & versions

        
      


    


    

      
First time here? Six questions answered


      

        Is this a paper, a tool, a dataset, or a company?
        
Not a company. It is three things: an open measurement protocol (Protocol v1.1 — what makes two submissions comparable), an MLCube container (anyone can reproduce the same measurement on their own GPU), and a dataset archived on Zenodo (360+ measured configurations, with a DOI); plus an SSRN preprint stating the conclusions. Single maintainer (Hongping Zhang), Apache-2.0. The findings are also reflected in Hugging Face Optimum's quantization documentation (based on dataset v1.1.0).

      


      

        Should I read the conclusions or the protocol first?
        
Conclusions first: spend a minute with the tool above to get a direction. You only need the protocol and the limitations when you are about to cite it or make a decision on it — the site deliberately splits “what we require / what we measured / what changed / what we are still exploring” into four layers so you never have to wade through methodology up front.

      


      

        I have no GPU — can I use this?
        
Yes. The estimator is pure front-end and needs no hardware at all. To get your own measured number, a free Colab T4 is enough (20–35 minutes, nothing to install) and produces the same energy.json as the container; both arms have to fit, which on a 16 GB T4 means models up to about 3B.

      


      

        Can these results guide a production deployment?
        
At this stage they give a direction, not a production decision. Every number is GPU-package power (NVML) — excluding PSU losses, CPU, DRAM, cooling and PUE — measured at single-stream batch 1 over 256 tokens, in the generation window only. It is neither whole-system power nor any certified benchmark result. For a production decision, measure your own stack, batch and workload — which is exactly what the container is for.

      


      

        I'm a researcher, a cloud provider, or a developer — where should I click?
        

          
Researcher / about to cite it → Protocol & number glossary · coverage matrix · DOI & BibTeX · limitations

          
Cloud vendor / platform team / anyone with a cluster → container, batch runs · contribute data · variable matrix (batch and context are entirely empty — that is the biggest gap)

          
Developer / just deciding whether to quantize → the estimator above · when not to quantize · ask Watt · or run Colab on your own model

        

      


      

        Does “NF4 saves energy” hold for my model?
        
It depends on your model size and your GPU — there is no unconditional answer. The fitted crossovers sit near Turing 2.1B / Ada 3.7B / Blackwell 4.8B: below the crossover quantization usually costs more energy, above it, it starts to save. And coverage is almost entirely bitsandbytes at batch 1, so changing the runtime can flip the sign — on the same card, llama.cpp Q4_0 is −61.9% while bitsandbytes LLM.int8() is +106%. So: get a direction from the estimator, then verify on your own stack with the container.

      

    


    

      
Full content index · how to read this site



      
Four layers · how to read this site

      
The site is deliberately split so that what we require,
      what we measured, what changed and what we are still exploring never blur into one page.

      

        

          
1 · Spec — the normative protocol

          
What makes two submissions comparable. The normative protocol, the container and schema that enforce it.

          
Protocol v1.1 · Number glossary · Container guide · Run it yourself · Schema

        

        

          
2 · Evidence — what we measured

          
Everything measured, at its replication level — and every count with its denominator.

          
Evidence hub · Coverage matrix · Latest: two-card NF4 re-test · Public replications · Cite

        

        

          
3 · Changelog — what changed

          
What changed in the data, protocol and site — dated, typed, linked to its evidence.

          
All entries · Window unification (09-24) · NF4 two-card re-test (09-25)

        

        

          
4 · Findings — what we are still exploring

          
Deeper write-ups and tools whose output is a model, not a measurement — never counted in the matrix.

          
Notes & findings · Variable matrix · Estimator · Optimizer

        

      

    


  



    
Recent updates

    

      2026-09-25 · data RTX 4090 NF4 re-tested across two physical cards — July's +0.8% anchor holds (mean +1.6%, n=3); the −15.1% historical reading was not reproduced and stays a divergent observation; no direction flip at the 3B anchor under the current stack. →

      2026-09-24 · data All energy numbers unified on the generation measurement window; the llama.cpp row re-cut from its 100 Hz traces (−62.9% at 576 tokens, counter basis). →

      2026-09-25 · site The site split into four layers — spec, evidence, changelog, findings; every count now has a defined denominator. →

      Full history, each entry linked to its evidence: the changelog →
    

  

  
    


    

      Public research layer. This site is static: documentation, charts, dataset, DOIs and reproducible
      materials — Apache-2.0, no login, no result tracking. A future service layer (benchmark API, report
      generation, organisation & project management, private measurement results, enterprise permissions)
      is defined as a contract only and is deliberately not deployed. The two layers share exactly one
      thing: the energy.json schema — which is what keeps any future private result comparable
      with the curves published here.