EcoCompute · Methodology

How EcoCompute measures LLM inference energy

Direct on-device power sampling, not TDP arithmetic. This page is the citable description of the protocol behind every number on quantenergy.tech.

In one paragraph. Energy per configuration (model × precision × GPU) is obtained by sampling GPU package power with NVML at 10 Hz while the model generates 256 tokens per run, repeating the run and integrating power over wall-clock time. Nothing is derived from thermal design power. Values that are not measured — sizes between or beyond our measured anchors, and latency/throughput — are modelled and labelled as interpolated, extrapolated or estimated in the UI and in the API response. Measurements and estimates are never mixed silently.

1 · Measurement protocol

Quantity	How it is obtained
GPU power	NVML (`nvmlDeviceGetPowerUsage`) polled at 10 Hz for the duration of generation; GPU-package power, board-level
Workload	Single-stream text generation, 256 output tokens per run, fixed prompt and sampling settings
Repeats	Main dataset (v1.1.0): n = 2 with CV < 2%. Container runs reported on the site: 10 iterations, CV < 3%
Energy	Time-integral of sampled power over the generation window, normalised to joules per 1M output tokens
Baseline	Each model's own FP16 run on the same GPU; ΔE% is relative to that baseline
Precisions	FP16, NF4, INT8 (FP8 where the hardware supports it) — weight-only quantization
Architectures	Turing (T4), Ampere (A800), Ada (RTX 4090 / 4090D), Blackwell (RTX 5090)

2 · What is measured vs modelled

Every result carries a basis field, and the site renders it as a tag next to the number:

measured — a real NVML run exists for exactly this configuration.
interpolated — the model size falls between two measured anchors on the same architecture and precision; the value comes from the fitted crossover curve (curves.json).
extrapolated — the size lies outside the measured range; treat the value as directional only.
estimated — latency, throughput and VRAM figures from a roofline-style model, not from a power measurement.

The fitted curves and the estimator are open: see estimate.js, optimize.js and the build/ scripts in the site repository.

3 · The finding this protocol produced

Weight-only quantization (NF4/INT8) reliably reduces memory footprint, but it does not always reduce energy. On small models the dequantization overhead outweighs the memory-bandwidth savings, so quantization increases energy per token; above a crossover point that depends on both model size and GPU architecture, it starts to save. Explore your own case on the Your model tab.

4 · Reproduce it on your GPU

The measurement code is a container, not a description. The EcoCompute energy MLCube runs the same NVML sampling on your hardware and writes an energy.json with the exact fields these charts read:

git clone https://github.com/hongping-zh/ecocompute-mlcube.git
cd ecocompute-mlcube

pip install mlcube mlcube-docker
mlcube run --mlcube=. --task=energy_estimate --platform=docker

Then drop the resulting energy.json on the Run it yourself tab to overlay your point on the crossover curve — the overlay is encoded in the URL, nothing is uploaded.

5 · Limitations

NVML reports GPU-package power, not whole-system draw: CPU, DRAM, PSU losses and cooling are excluded.
Single-stream generation only; batched serving with continuous batching has a different energy profile.
The RTX 4090 (Ada) July 2026 column is n = 1 per configuration — a supplementary deep dive, not a replacement for the main dataset.
This is a research protocol, not a certified benchmark, and not affiliated with any energy-consulting body.

6 · How to cite

The methodology and argument — preprint “Weight-Only Quantization Does Not Always Save Energy: An Empirical Study of LLM Inference Across NVIDIA GPU Platforms”, SSRN #6854700 (archived: 10.5281/zenodo.21066652).
The numbers on this site — main dataset v1.1.0, 10.5281/zenodo.19647290 (CC BY 4.0).
The RTX 4090 (Ada) deep dive — 10.5281/zenodo.21528103 (CC BY 4.0).