EcoCompute · Public replications

Don't take our word for it — reproduce it on your GPU.

Independent replications of the EcoCompute energy measurements, published as they arrive.

Our central claim — weight-only quantization doesn't always save energy — rests on direct NVML power measurements from a handful of cards run by one maintainer. That is exactly as strong as its first independent confirmation. Or its first credible disagreement: we publish both, in the same gallery, with the same weight.

Run the open MLCube container on your card, drop the resulting energy.json below, and see your point land on — or off — the published crossover curve. The file is parsed in your browser; this site has no upload endpoint, so nothing is submitted until you paste the summary into a GitHub issue and send it yourself.

See replications ▼ Run yours (60s) →

MLCube-compatible Schema-checked in your browser Apache-2.0 Moderated for format, never for results Disagreements welcome

How it works

The same three steps as the Run it yourself tab — the third one makes your result public.

1 · Run the container

Docker + an NVIDIA GPU. No GPU? It still completes, but the report is derived from the published dataset and labelled basis ≠ measured — never faked, and not eligible for the gallery as a measurement.

2 · Get energy.json

Two fields the charts read: results.vs_fp16_energy_pct (signed ΔE% against the same model's own FP16 run on the same card) and results.basis (measured / interpolated / extrapolated).

3 · Submit & overlay

Your point joins the crossover chart and this gallery, credited to your handle plus optional affiliation. You can withdraw it at any time by asking on GitHub — no questions, no email needed.

Why bother? Every independent measurement — agreement or disagreement — makes the published curve more useful to the next person deciding whether to quantize. Contributors are credited by handle on every chart their point appears on.

Submit a replication

Parsing happens entirely in your browser. When you are ready, the button below copies the summary you see here, plus your raw report, to your clipboard and opens an empty GitHub issue — you paste it, read it, and send it. Nothing leaves this page on its own.

⬇ Drop your energy.json here, or click to choose a file

file: — schema: — basis: —

Handle / name (required)

Affiliation (optional)

Independent trials, n (you tell us)

Link (optional)

The submission text carries your whole energy.json, which is more than a GitHub URL can hold, so the button puts it on your clipboard and opens an empty issue — paste it there with Ctrl+V. Nothing is prefilled through the URL, so there is no length limit to hit.

GPU, model, precision, ΔE% and basis are read from the file — we do not let you retype them. n is not in the report: the container records measurement.iterations (decode iterations inside one run), which is not the same as repeating the whole measurement. If you ran the container once, n = 1, and your point is drawn hollow like our own single-trial data.

I confirm this is a real run on the hardware named in the file. I understand the submission will be public and credited to my handle. (Optional) Reviewers may follow up in the GitHub issue — no email required.

Requires a file, a handle, and the first two boxes.

Withdrawal and privacy, precisely. This page stores nothing: there is no server here to store it — the site is static, and your file never leaves your browser. What becomes public is the GitHub issue you choose to send, so withdrawal is a message on the same issue tracker referencing your handle; the entry is removed as soon as the maintainer sees it, normally within one working day, no questions asked. We never ask for your email. We do not claim zero logging: GitHub and this site's static host/CDN keep their own access logs, which we neither control nor read.

Replications gallery

First independent replication — thank you, @gkgoing

On 23 August 2026 the first energy.json measured by somebody other than the maintainer arrived: TinyLlama-1.1B in NF4 on an RTX 3050 Ti Laptop GPU, Windows, 39.8 W average package power — and it saved 16.2% of the energy at a model size where every card in our own set spends 4.6–33% more. Submission → · what it changes →

It also carried a paired perplexity — 10.1016 against an FP16 baseline of 9.6201 — which reproduces our own RTX 4090 measurement of the same model to four decimal places. The quality axis replicated exactly; the energy axis did not replicate at all. That contrast is the point of the whole project, and we did not have independent evidence for it until this run.

Honesty guardrails

Our measured anchors and community submissions are visually distinct on every chart — and so are single trials: filled marker = n ≥ 2 repeated trials, hollow marker = n = 1, the same convention used for our own data (see methodology).
Submissions are shown as provided, with the basis declared in their report.
We moderate for schema and format only, never for results. A run that contradicts the published curve is published as prominently as one that confirms it — and, if it holds up, it changes the curve.
Non-measured points (interpolated / extrapolated) are labelled and never presented as measurements — the same rule as the rest of the site.
“Consistent” / “outside the band” is computed here from the same 95% band the estimator uses — the fit's own instability (residual spread and leave-one-out error, curves.json), widened outside the fitted size range — not from anyone's judgement, and it is a statement about our fit, not about whose measurement is right. Where that band is very wide, the entry says so instead of claiming a confirmation.
A contributed point is drawn on the charts but never fitted: the published curves move only after review, and never silently. Nothing in curves.json or build/measured.csv changes when a replication is published.
Not yet built: signed reports. The container has no --sign task today, so there is no cryptographic “verified” badge — we would rather have no badge than a decorative one. Attach a link to a signed or logged run if you have one and we will link it.