The EcoCompute energy MLCube

EcoCompute is an open reproducibility project measuring how LLM quantization affects inference energy per token across GPUs and model sizes. This page is about the part you can run yourself: the container that produced those measurements.

中文读者：逐步的容器使用手册见 EcoCompute 容器数据测量 · 中文使用手册。

1 · Run it — one command, zero configuration

Prerequisite for a real measurement: an NVIDIA GPU with NVML power telemetry. Docker is optional — see the two modes below. Without a GPU you can still verify the whole pipeline — see step 3.

curl -fsSL https://raw.githubusercontent.com/hongping-zh/ecocompute-mlcube/main/quickstart.sh | bash

It measures NF4 against its own FP16 baseline on TinyLlama-1.1B, writes a schema-validated energy.json into ./ecocompute-out/, and prints the overlay link. The script prints every command it runs.

Time: the measurement is a few minutes; the first run is dominated by downloads and everything is cached afterwards. The one end-to-end run we have timed took 57 minutes on a China-hosted rented RTX 4090 in native mode — 5.5 of those were the ~2 GB model, nearly all the rest was ~3 GB of torch/CUDA wheels from download.pytorch.org. The setup script now prefers the host's configured (domestic) package index for torch, which should remove most of that, but we have not re-timed it, and we have not timed the docker path at all.

It picks one of two modes and tells you which:

• docker — when the Docker daemon answers: pulls the prebuilt image and runs it with --gpus all; nothing is installed on the host.
• native — when there is no usable Docker. Rented GPU instances (AutoDL, vast.ai and similar) are themselves containers and cannot nest one, so the script clones the repository and installs into a venv, reusing the host's CUDA PyTorch and keeping the checkout, venv and model cache on the data disk when there is one. Same entrypoint.py, same report.

Force either mode with ECOCOMPUTE_MODE=docker or ECOCOMPUTE_MODE=native.

You do not pass an architecture: gpu_arch defaults to auto and is derived from the NVML device name, so a run cannot label an RTX 4090 as Blackwell and end up compared against the wrong curve. A card we do not recognise is recorded by its NVML name rather than guessed into a family. Override anything via environment variables — ECOCOMPUTE_PRECISION=INT8, ECOCOMPUTE_MODEL + ECOCOMPUTE_PARAMS_B, ECOCOMPUTE_ITERATIONS (default 10 after 2 warmups, matching the published dataset). Both modes install the same pinned transformers/bitsandbytes as the image — quantization kernels change between releases, so this is what keeps contributed runs comparable.

Or drive the image directly:

docker run --rm --gpus all \
    -v "$PWD/out:/workspace/outputs" \
    ghcr.io/hongping-zh/ecocompute-mlcube:latest energy_estimate \
    --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --params_b 1.1 \
    --precision NF4 --gpu_arch auto --output_dir /workspace/outputs --share

The image is built and pushed by CI on every merge to main. If the pull is denied or you are offline, quickstart.sh says so and builds the image from the repository instead — same result, 10–20 minutes longer.

From source

git clone https://github.com/hongping-zh/ecocompute-mlcube.git
cd ecocompute-mlcube

# with the official MLCommons MLCube CLI (task name: energy_estimate)
pip install mlcube mlcube-docker
mlcube run --mlcube=. --task=energy_estimate --platform=docker

# ...or drive the entrypoint directly (development / CI)
python3 entrypoint.py energy_estimate \
    --parameters_file workspace/parameters/energy_params.yaml \
    --output_dir workspace/outputs

cat workspace/outputs/energy.json

Swap the run without editing defaults by pointing at another parameters file: mlcube run --mlcube=. --task=energy_estimate parameters_file=parameters/bert_bs32.yaml output_dir=outputs_bert/.

Compare against this site straight from the terminal

python3 entrypoint.py energy_estimate \
    --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
    --precision NF4 --gpu_arch auto --params_b 1.1 \
    --output_dir workspace/outputs --prefetch --share

• --prefetch — asks the site estimator for its prediction before measuring and prints it, so you see prediction vs. measurement side by side. Offline or slow? It is skipped silently; the measurement is never blocked or altered.
• --share — prints (and writes share_url.txt) a link of the form quantenergy.tech/?tab=run&overlay=…. Your point is encoded in the URL itself: opening it restores your marker on the crossover curve entirely in the browser. Nothing is uploaded and no server stores anything.

2 · What comes out

One JSON report per configuration, validated against schema/energy.schema.json. The two fields this site plots are results.vs_fp16_energy_pct and results.basis:

Sample outputs ship with the repository: examples/energy.no-gpu.json and examples/energy.measured.illustrative.json (the latter's numbers are synthetic and labelled as such — it documents the shape of the measured path, not a run).

3 · Honesty guarantees

Determinism matters here for a specific reason: bitsandbytes NF4/INT8 kernels and torch change their numeric behaviour between releases. Top-level dependencies are pinned with exact == versions and the image installs from a full transitive lock captured from the verified CUDA image.

4 · Verified with the official MLCube CLI

The container was exercised with MLCommons' mlcube CLI (v0.0.9, docker platform):

• mlcube describe accepts the descriptor: task energy_estimate, inputs=[parameters_file], outputs=[output_dir].
• mlcube run on the CPU descriptor builds the image, mounts the workspace and writes workspace/outputs/energy.json.
• A parameter swap produces a genuinely different report (bert-base-uncased, batch_size=32 → Offline scenario).
• Both reports validate against the JSON Schema; the pytest suite drives the task through its real CLI and asserts that the no-GPU path never labels a result measured.

5 · The runs on this site came out of it

The newest column here — RTX 4090 (Ada), July 2026, five models from 0.5B to 7B × FP16 / NF4 / INT8, all fifteen configurations real hardware measurements — was produced by this container, driven by the sweep scripts in autodl/ (setup → environment verification → full model × precision sweep → aggregation). That run extended Ada coverage to INT8 and to 7B for the first time, and it is archived as its own citable artifact: 10.5281/zenodo.22037483 (CC BY 4.0). It is a supplementary deep dive alongside the main v1.1.0 dataset, not a replacement: n = 1 per configuration.

Full protocol behind both: Methodology. Artifact map and DOIs: Artifacts & citation.

6 · Scope — what this is not

• No LoadGen. The scenario field is a nominal label derived from batch size; the container runs its own warmup/iterations loop and applies no MLPerf LoadGen timing constraints.
• No accuracy target. Only energy and throughput are reported — quantization quality is out of scope.
• NVML measures GPU-package power, not wall power: CPU, DRAM, PSU losses and cooling are excluded.
• This is a supplemental energy-methodology container and not a certified MLPerf result; every report carries certified_benchmark_result: false.

7 · Not built yet

So the roadmap is not mistaken for the product — these are wanted, but they do not exist today:

• Multi-architecture and multi-GPU images. The published image is linux/amd64, single-GPU (accelerator_count: 1); it does not measure multi-card or tensor-parallel serving.
• A wall-power hook (BMC / IPMI / smart PDU / USB meter) to report system energy alongside the NVML number. This is the most common and most fair critique of NVML-only measurement.
• Signed reports for ESG and procurement contexts.
• A one-task reproduction of the full published anchor set, and a public wall of community reproductions.

8 · Contribute a measurement

We are looking for collaborators to reproduce EcoCompute measurements across additional GPUs, cloud instances, model families and quantization backends. If you operate NVIDIA, AMD or other accelerator infrastructure and want to contribute comparable energy-per-token data, please get in touch.

Most valuable, in order:

1. Runs that contradict the published curve. A disagreement with a schema-valid energy.json attached is the single most useful thing you can send.
2. GPUs we have never measured — RTX 3090, L4, A100, H100, MI300X and friends.
3. Backends beyond bitsandbytes — AWQ, GPTQ, FP8, vendor runtimes.
4. Wall-power implementations for your BMC or PDU.

How: use the submission form — it parses your energy.json in your browser, previews your point against the fitted curve, then copies the submission to your clipboard and opens an issue you paste into and send yourself. You can also open an issue or pull request directly on the container repository with your energy.json (and the --share link if you have one), or reach the maintainer via hongping-zh.github.io. Accepted submissions appear in the public replications gallery, moderated for schema and format only — never for results.

9 · Cite the container

@software{ecocompute_mlcube,
  title   = {EcoCompute energy MLCube: reproducible LLM inference energy measurement},
  author  = {Zhang, Hongping},
  year    = {2026},
  url     = {https://github.com/hongping-zh/ecocompute-mlcube},
  license = {Apache-2.0}
}

Cite the datasets and the preprint for the numbers and the argument; cite the container for the measurement procedure. The container has no Zenodo DOI yet — until it does, cite the repository and pin the commit you ran.