The EcoCompute energy MLCube

EcoCompute is an open reproducibility project measuring how LLM quantization affects inference energy per token across GPUs and model sizes. This page is about the part you can run yourself: the container that produced those measurements.

中文读者：逐步的容器使用手册见 EcoCompute 容器数据测量 · 中文使用手册。

0 · What the container guarantees

Where this release sits — the version matrix

“Running the project” means the container row: release schema-1.3-r3, whose reports validate as ecocompute-energy/1.3 against Protocol v1.1. The full version-by-version correspondence lives in the protocol’s §8.

1 · Run it — one command, zero configuration

Prerequisite for a real measurement: an NVIDIA GPU with NVML power telemetry. Docker is optional — see the two modes below. Without a GPU you can still verify the whole pipeline — see step 3.

No GPU at all? The Colab notebook (source) runs the same NVML measurement on a free T4 and emits the same schema-validated energy.json. It is a re-implementation of entrypoint.py's sampler rather than the container itself, so it is not pinned to the image's dependency set, and it measures both arms in randomized order with a cooldown to idle before each — a protocol the container does not enforce. On a 16 GB T4 both arms fit up to about 3B parameters.

curl -fsSLO https://github.com/hongping-zh/ecocompute-mlcube/releases/download/schema-1.3-r3/quickstart.sh
curl -fsSLO https://github.com/hongping-zh/ecocompute-mlcube/releases/download/schema-1.3-r3/SHA256SUMS
sha256sum -c SHA256SUMS
bash quickstart.sh

It measures NF4 against its own FP16 baseline on TinyLlama-1.1B, writes a schema-validated energy.json into ./ecocompute-out/, and prints the overlay link. The script prints every command it runs — and bash quickstart.sh --dry-run prints the whole plan (mode, every directory it would touch, the exact commands) and exits without doing anything.

Why four lines instead of curl | bash: piping a moving branch into a shell means executing whatever main holds at that moment, with no version, no checksum and no signature. Here you download from an immutable release, verify the SHA-256 against the published SHA256SUMS, and only then execute. The same release also carries a CycloneDX SBOM (from the pinned requirements.lock.txt) and the image digest below. Two more properties travel with the script: the container never runs as root — the image's default user is unprivileged and the script always passes --user $(id -u):$(id -g) — and ECOCOMPUTE_READONLY=1 optionally adds --read-only plus a tmpfs /tmp, leaving only the two bind mounts writable. What the script changes on your host is written out in full in its header: the output dir, the model cache, and (native mode only) a checkout + venv on the data disk — nothing else.

Pin the image by digest (instead of the mutable :latest) for a fully reproducible run:

ECOCOMPUTE_IMAGE=ghcr.io/hongping-zh/ecocompute-mlcube@sha256:5c812e66c34fcdab436bdcf4c1135dc91a6a86b1f94d3efb6118947a7492e275 \
ECOCOMPUTE_REF=schema-1.3-r3 bash quickstart.sh

The release page lists the full SHA-256 values of the schema, the validator, the quickstart and the SBOM, plus the release commit and the CI build that produced the image.

Time: the measurement is a few minutes; the first run is dominated by downloads and everything is cached afterwards. The one end-to-end run we have timed took 57 minutes on a China-hosted rented RTX 4090 in native mode — 5.5 of those were the ~2 GB model, nearly all the rest was ~3 GB of torch/CUDA wheels from download.pytorch.org. The setup script now prefers the host's configured (domestic) package index for torch, which should remove most of that, but we have not re-timed it, and we have not timed the docker path at all.

It picks one of two modes and tells you which:

• docker — when the Docker daemon answers: pulls the prebuilt image and runs it with --gpus all; nothing is installed on the host.
• native — when there is no usable Docker. Rented GPU instances (AutoDL, vast.ai and similar) are themselves containers and cannot nest one, so the script clones the repository and installs into a venv, reusing the host's CUDA PyTorch and keeping the checkout, venv and model cache on the data disk when there is one. Same entrypoint.py, same report.

Force either mode with ECOCOMPUTE_MODE=docker or ECOCOMPUTE_MODE=native.

You do not pass an architecture: gpu_arch defaults to auto and is derived from the NVML device name, so a run cannot label an RTX 4090 as Blackwell and end up compared against the wrong curve. A card we do not recognise is recorded by its NVML name rather than guessed into a family. Override anything via environment variables — ECOCOMPUTE_PRECISION=INT8, ECOCOMPUTE_MODEL + ECOCOMPUTE_PARAMS_B, ECOCOMPUTE_ITERATIONS (default 10 after 2 warmups, matching the published dataset). Both modes install the same pinned transformers/bitsandbytes as the image — quantization kernels change between releases, so this is what keeps contributed runs comparable.

Or drive the image directly (the default user is unprivileged; passing --user makes the report land in a directory you own):

mkdir -p out
docker run --rm --gpus all --user "$(id -u):$(id -g)" \
    -v "$PWD/out:/workspace/outputs" \
    ghcr.io/hongping-zh/ecocompute-mlcube:latest energy_estimate \
    --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --params_b 1.1 \
    --precision NF4 --gpu_arch auto --output_dir /workspace/outputs --share

The image is built and pushed by CI on every merge to main. If the pull is denied or you are offline, quickstart.sh says so and builds the image from the repository instead — same result, 10–20 minutes longer.

From source

git clone https://github.com/hongping-zh/ecocompute-mlcube.git
cd ecocompute-mlcube

# with the official MLCommons MLCube CLI (task name: energy_estimate)
pip install mlcube mlcube-docker
mlcube run --mlcube=. --task=energy_estimate --platform=docker

# ...or drive the entrypoint directly (development / CI)
python3 entrypoint.py energy_estimate \
    --parameters_file workspace/parameters/energy_params.yaml \
    --output_dir workspace/outputs

cat workspace/outputs/energy.json

Swap the run without editing defaults by pointing at another parameters file: mlcube run --mlcube=. --task=energy_estimate parameters_file=parameters/bert_bs32.yaml output_dir=outputs_bert/.

Cold or steady: which card you measured

A GPU measured straight off idle and the same GPU measured after ten minutes of load are two different machines: the hot one clocks lower and draws more, so it reports more energy per token. Since protocol ecocompute-protocol/1.1 the report says which one you measured.

--thermal_mode cold     # default: a fixed warmup, then measure — matches every published EcoCompute number
--thermal_mode steady   # warm up until the die temperature stops rising, then measure

steady runs warmup passes one second apart until three consecutive temperature readings fall within 2 °C, then starts sampling; warmup energy is never counted in the result. If the budget runs out first the report says steady_state_reached: false and keeps the temperatures it did observe, rather than claiming a steady state it never reached. The two modes are comparable within a mode and not across one — and the 42 seed measurements predate the field entirely, so their thermal state is unknown, not cold. That is why the default stayed cold.

Compare against this site straight from the terminal

python3 entrypoint.py energy_estimate \
    --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
    --precision NF4 --gpu_arch auto --params_b 1.1 \
    --output_dir workspace/outputs --prefetch --share

• --prefetch — asks the site estimator for its prediction before measuring and prints it, so you see prediction vs. measurement side by side. Offline or slow? It is skipped silently; the measurement is never blocked or altered.
• --share — prints (and writes share_url.txt) a link of the form quantenergy.tech/?tab=run&overlay=…. Your point is encoded in the URL itself: opening it restores your marker on the crossover curve entirely in the browser. Nothing is uploaded and no server stores anything.

2 · What comes out

One JSON report per configuration, validated against schema/energy.schema.json. The two fields this site plots are results.vs_fp16_energy_pct and results.basis:

Sample outputs ship with the repository: examples/energy.no-gpu.json and examples/energy.measured.illustrative.json (the latter's numbers are synthetic and labelled as such — it documents the shape of the measured path, not a run).

3 · Honesty guarantees

Determinism matters here for a specific reason: bitsandbytes NF4/INT8 kernels and torch change their numeric behaviour between releases. Top-level dependencies are pinned with exact == versions and the image installs from a full transitive lock captured from the verified CUDA image.

4 · Verified with the official MLCube CLI

The container was exercised with MLCommons' mlcube CLI (v0.0.9, docker platform):

• mlcube describe accepts the descriptor: task energy_estimate, inputs=[parameters_file], outputs=[output_dir].
• mlcube run on the CPU descriptor builds the image, mounts the workspace and writes workspace/outputs/energy.json.
• A parameter swap produces a genuinely different report (bert-base-uncased, batch_size=32 → Offline scenario).
• Both reports validate against the JSON Schema; the pytest suite drives the task through its real CLI and asserts that the no-GPU path never labels a result measured.

5 · The runs on this site came out of it

The newest column here — RTX 4090 (Ada), July 2026, five models from 0.5B to 7B × FP16 / NF4 / INT8, all fifteen configurations real hardware measurements — was produced by this container, driven by the sweep scripts in autodl/ (setup → environment verification → full model × precision sweep → aggregation). That run extended Ada coverage to INT8 and to 7B for the first time, and it is archived as its own citable artifact: 10.5281/zenodo.22037483 (CC BY 4.0). It is a supplementary deep dive alongside the main v1.1.0 dataset, not a replacement: n = 1 per configuration.

Full protocol behind both: Protocol. Artifact map and DOIs: Artifacts & citation.

6 · Scope — what this is not

• No LoadGen. The scenario field is a nominal label derived from batch size; the container runs its own warmup/iterations loop and applies no MLPerf LoadGen timing constraints.
• No accuracy target. Only energy and throughput are reported — quantization quality is out of scope.
• NVML measures GPU-package power, not wall power: CPU, DRAM, PSU losses and cooling are excluded.
• This is a supplemental energy-methodology container and not a certified MLPerf result; every report carries certified_benchmark_result: false.

7 · Not built yet

So the roadmap is not mistaken for the product — these are wanted, but they do not exist today:

• Multi-architecture and multi-GPU images. The published image is linux/amd64, single-GPU (accelerator_count: 1); it does not measure multi-card or tensor-parallel serving.
• A wall-power hook (BMC / IPMI / smart PDU / USB meter) to report system energy alongside the NVML number. This is the most common and most fair critique of NVML-only measurement.
• Signed reports for ESG and procurement contexts.
• A one-task reproduction of the full published anchor set, and a public wall of community reproductions.

8 · Contribute a measurement

We are looking for collaborators to reproduce EcoCompute measurements across additional GPUs, cloud instances, model families and quantization backends. If you operate NVIDIA, AMD or other accelerator infrastructure and want to contribute comparable energy-per-token data, please get in touch.

Most valuable, in order:

1. Runs that contradict the published curve. A disagreement with a schema-valid energy.json attached is the single most useful thing you can send.
2. GPUs we have never measured — RTX 3090, L4, A100, H100, MI300X and friends.
3. Backends beyond bitsandbytes — AWQ, GPTQ, FP8, vendor runtimes.
4. Wall-power implementations for your BMC or PDU.

If you have no GPU to spare, the Colab notebook produces a submittable report on a free T4 in well under an hour.

How: use the submission form — it parses your energy.json in your browser, previews your point against the fitted curve, then copies the submission to your clipboard and opens an issue you paste into and send yourself. You can also open an issue or pull request directly on the container repository with your energy.json (and the --share link if you have one), or reach the maintainer via hongping-zh.github.io. Accepted submissions appear in the public replications gallery, moderated for schema and format only — never for results.

9 · Cite the container

@software{ecocompute_mlcube,
  title   = {EcoCompute energy MLCube: reproducible LLM inference energy measurement},
  author  = {Zhang, Hongping},
  year    = {2026},
  url     = {https://github.com/hongping-zh/ecocompute-mlcube},
  license = {Apache-2.0}
}

Cite the datasets and the preprint for the numbers and the argument; cite the container for the measurement procedure. The container has no Zenodo DOI yet — until it does, cite the repository and pin the commit you ran.