What changed, and when
Every change to the protocol, the data, published findings and the site itself —
newest first, each entry linking to the evidence it rests on. Entries older than this page live in the
notes and the dataset release notes on Zenodo; nothing before 2026-09-03 was retroactively
reconstructed.
2026-11-13site
The differential method now has a drawing: an interactive isometric bench at /bench/
The README has explained the differential method — measure an FP16 arm and a quantized arm of the same
workload on the same card in the same session; the difference is the quantization's net energy effect, never the
inference cost itself, and idle is never subtracted — in prose since the beginning.
It now also explains it in one picture: /bench/ is a self-contained interactive isometric
diagram of a differential power bench — two engines, one per arm, feeding a central manifold; a flywheel that
actually spins, cams that actually push, a chart recorder that actually plots; live readouts; keyboard control
(arrow keys switch arms, space restarts the strip, escape cuts the power; click or tap cycles the arms). One
file serves both themes: it follows the system setting, the footer switch or ?theme=light overrides
it, and the old /bench/light.html URL redirects to the override. The drawing is
generated from the same model as the container and its geometry is machine-checked on every rebuild — but by a
drawing toolchain that is not part of this repo, so unlike the container's validator that check is not
independently reproducible, and we no longer quote its counts here. What is verifiable is where the demo
numbers come from: the measured ada / NF4 / 7.0B cell (-33.7%, n=2) of data/coverage_matrix.csv. A
550 KB single file, no external requests, works offline. Entry point on the container
page.
2026-11-11sitedata
The grid grows: Volta enters the coverage matrix — 26/56 is now 29/64
The three V100S runs above are now cells in the coverage matrix, not just gallery entries: the matrix's own rule is "every published direct NVML measurement", and the Volta cells meet it the same way the maintainer cells do — vs a same-card FP16 baseline, no conversion. Denominator 56 → 64 (one new row), filled 26 → 29. The row is annotated, not laundered: each cell reads n=1 · t=c — the first cold-tagged cells on the grid, produced natively by the post-schema-1.3 container — and the caption states the disclosed torch deviation (2.6.0+cu124, no sm70 wheel in the pinned stack). Same treatment as the 5090 9-20 session: a deviation from the pins is shown, not hidden, and nothing here is fitted.
Count hygiene swept with it: the stat cards on /measured/ split the ambiguous "5 + 1" into 5 arch. classes measured (glossary-defined: ≥1 direct NVML measurement, maintainer or community — borrowed families like Hopper never count) and 2 / 5 independent replicators. The matrix CSV is renamed to the neutral data/coverage_matrix.csv (v2 layout: per-cell provenance — @handle + issue link — plus generated_at / source_commit / schema_version header lines; supersedes coverage_matrix_2026-09-09.csv), and the data index now registers the three Volta reports.
2026-11-10datafinding
First Volta-architecture data: three community runs on a V100S, all three sizes saving energy
The second independent contributor, @OliverYanghoo, submitted three NF4 runs in one morning on a Tesla V100S-PCIE-32GB — TinyLlama-1.1B (−1.7%), Qwen2.5-3B (−7.9%) and Qwen2.5-7B (−3.6%) (#42, #43, #44). Volta is the first architecture class measured entirely outside the maintainer's own hardware, and the sign pattern — savings at every size, matching the Ampere-laptop replication and opposite to the fitted Ada/Blackwell consumer curves — is the first architecture-class-level support for the bandwidth-starvation hypothesis from an external contributor. All three are in the gallery and the coverage matrix now has a Volta row. They stay out of the fitted curves for now: n = 1 per point and a disclosed software deviation (torch 2.6.0+cu124 — the pinned CUDA-13 stack ships no sm70 kernel for Volta), which the submission documented precisely, along with a real bug: --gpu_arch auto misses "V100S" because the \bv100\b pattern has no word boundary after the digits.
2026-11-09site
The verdict ladder is now visible: a homepage explainer and the first dataset-eligible report on display
The container's validator (tools/validate.py) has graded every energy.json on three rungs — schema-valid → protocol-conformant → dataset-eligible — since it shipped, but the ladder lived only in the repo. Now the Run-it-yourself path on the homepage explains what each rung checks and what it does not (an honest report can sit at rung 1: our own cold arm records warmup: 0 as a disclosed deviation), and the replications page shows the first real measurement to carry the top verdict — the steady-state RTX 4090 arm of the cold–steady thermal experiment, zero violations in both profiles, with its two disclosures stated plainly: it is the maintainer's own run (not an independent replication), and five protocol fields were backfilled from the operator's external temperature log before the container emitted them natively (ecocompute-mlcube #29, closed the day before).
2026-11-08site
The homepage's structured data now carries the container's Zenodo DOIs
Every DOI-backed entity in the homepage's JSON-LD graph — the paper, the main dataset, the RTX 4090 dataset — declared its identifiers except one: the EcoCompute energy MLCube node pointed at its GitHub repository but not at its Zenodo software record, even though the container page's visible BibTeX cites it. An external review of the site's machine-readable metadata caught the gap. The node now carries both the version DOI (10.5281/zenodo.23974826, schema-1.3-r5 — the same version-first convention as the other nodes) and the concept DOI (10.5281/zenodo.23974825). Nothing visible on the page changes; this is for crawlers, citation tools and the audit trail.
2026-11-08datafinding
The cold–steady gap now has a measured size: +4.1% per token, from a six-arm controlled experiment
The protocol page has said "compare within a mode, not across one" since v1.1 — until now, as a design
principle without its own number. It has one. Six runs on a single RTX 4090 (TinyLlama-1.1B, NF4, batch 1,
greedy decoding, 10 × 256-token generations, direct NVML at 10 Hz, zero dropped samples): three cold arms
launched at 37–39 °C and three steady arms at a 53–54 °C plateau confirmed by an external
5-second nvidia-smi log — 2033.8 ± 25.2 vs 2116.3 ± 33.0 mJ/token, +4.1%, monotonic across an
intermediate 47 °C arm that was discarded for insufficient cooldown but documented in the run notes.
Both the method page's thermal-mode bullet and its Limitations list now carry the
number instead of the principle.
Disclosures that travel with it: a different physical card than the main dataset's
4090D; bitsandbytes 0.50.2 instead of the pinned 0.43.3 (CUDA 13 kernel availability — every report carries
differs_from_reference_pins); the r5 tool emits null for three thermal fields, backfilled
post hoc from the external log with the pristine originals archived alongside; steady arms validate
protocol-conformant, cold arms retain one intentional, disclosed deviation (warm-up = 0 by design — the
cold-start case current warmup-mandating protocols have no path for). Published as its own dataset, not
pooled into the curves: 10.5281/zenodo.24046432
(CC BY 4.0). No existing number changed.
2026-11-07container
Supply-chain honesty pass: the SBOM claim matched to reality, dependabot barred from the reproducibility lock, and the release checklist gains an SBOM step
An external review of the container repo raised two supply-chain findings; both survived fact-checking. First: the README claimed “SBOM included”, but only the r3 release actually carries sbom.cdx.json — r4 and r5 shipped without one, and the repo tree (and therefore the Zenodo r5 archive) has no SBOM at all. The README now says exactly that: “SBOM release asset (r3; standard again from r6)” (f75b21f). Second: dependabot was opening weekly bump PRs against requirements.lock.txt — the fully-pinned reproducibility asset whose own header says it exists “so quantization-backend behaviour … is frozen to a known-good set” — including a huggingface-hub 0.36.2→2.4.0 jump across two major versions: the automated version of the torch-mismatch that once invalidated INT8 data. The pip ecosystem is now removed from dependabot.yml (211ca2b); the five open bump PRs were closed with that rationale, and dependency refreshes now happen only at release boundaries, after in-container NF4/INT8 re-validation. The CONTRIBUTING release checklist now requires regenerating the SBOM against the published image for every release (0f55c62) — the same move as the 2026-10-28 CRLF CI guard: a discipline that lived in a comment becomes a rule that lives in the checklist. r6 will be the first release shipped under it.
2026-11-05site
Container release schema-1.3-r5: the r5/r6 bundle shipped together; site pins updated
Release schema-1.3-r5 (2026-11-05) is out, and it bundles everything main had accumulated since r4: the engine-identity block with token-count acceptance conditions and the reference fake adapter (PR #26), the AMD-fallback NVML-first cross-check (PR #28), and a mock-tested ROCm power/temperature telemetry sampler (PR #31, @amarjithkumar2007-wq — not yet wired into the measurement flow). The 2026-10-29 audit entry said the engine block was “schema 1.3-r6 content waiting for its tag-and-release ceremony”: in the event, r5 shipped both planned releases’ content at once, so r6 never got its own tag. quickstart.sh, the schema and the validator are byte-identical to r4 (SHA256SUMS.all). The homepage and container-page version matrices now pin r5, and their “main is ahead” notes are replaced by the actual release contents; the broken /spec/#s8-version-correspondence anchors (the §8 table lives in the protocol document, not the /spec/ index) now link to the right file. The protocol page’s §8 correspondence rows and its §2 machine-contract pin move to r5; the dated §5.1 pins (2026-10-16, r4) stand as history. The release also carries the project’s Zenodo software-DOI metadata (.zenodo.json, CITATION.cff), with Amarjith Sah listed as co-author on the container’s software record.
2026-10-29site
New blog post: "Zero code, two releases: a bug report was the contribution"
The 2026-10-28 fix, told end-to-end: a paid reproducer whose script never ran past line 1, a verification report that passed all ten checks, and one remark in it — "the quickstart header still says r3" — that traced to the README's last live CRLF path. The post's thesis: a failed run, a footnote and a sentence of confusion are contributions — the scarce kind for a single-maintainer project — and the CONTRIBUTING release checklist now enforces the dry-run rule his bug bought. Read it here.
2026-10-29site
A five-point post-fix audit: two findings survived fact-checking and changed the site; three did not
An audit pass over yesterday's CRLF fix raised five concerns; fact-checking narrowed them to two site changes, both in this entry. (1) The version matrices on the homepage and the container page say schema-1.3-r4 — numerically correct, because r4 is still the latest tagged release, but the repo's main is already ahead of it: PR #26 and #28 merged the engine-identity block (schema 1.3-r6 content), which has no tag or release yet. Both matrices now carry a one-sentence "main is ahead" note so a repo reader is not confused by the gap; the r5/r6 tag stays deliberately unpinned until the release ceremony. (2) The container page's one-liner paragraph now mentions --dry-run and links the stub and the wrapper for pre-run reading, matching the homepage's security-model note. The other three findings, recorded for the audit trail: "only run.sh and quickstart.sh were fixed for CRLF" was a false alarm — all 21 shell/Python files on main are LF, with the .gitattributes rule and the CI CRLF guard in place; the GPU test gap is real but already stated ("GPU reproduction is queued"), and its result will be logged here when it happens; crediting the reproducer by name waits for their consent, which has been asked in the maintainer's reply.
2026-10-28site
Trademark notice audit: every page that names MLCube / MLCommons / MLPerf now carries the nominative-use notice
A full-site audit found the trademark notice (the sentence pinned on 2026-10-27 as a standing rule for every channel that names these marks) was present on some pages but absent on others that name them just as often: 18 pages — including the homepage, the cite page, the contribute page, the protocol document and the changelog itself — had no notice at all, and four blog footers used a drifted wording. Fixed in one pass: all 18 now carry the canonical notice ("MLCommons, MLPerf and MLCube are trademarks of MLCommons Association, used nominatively.") in their footer, the four blog footers are unified to the same wording, and the maintainer's GitHub profile README — a channel that names all three marks — gained the notice as well. Pages whose existing notice was a fuller variant (about, container, mlperf-client) were left alone: they already state nominative use plus non-affiliation. Nothing on any page changed except the notice line.
2026-10-28site
The reproducer's v2 verification passed all 10 checks — and chasing one remark in it exposed the last live path to the CRLF bug: the README's quick start still pointed at r3
The paid reproducer's second report verified the morning's CRLF fix item by item (stub, wrapper, r4 asset, tag-level hashes, the corrected schema hash independently recomputed from the tag) — all green, GPU reproduction queued. One remark in it ("the quickstart header still says r3") did not reproduce against the live r4 asset, but chasing where an r3 could come from found a real one: the README's quick start download URLs still pinned schema-1.3-r3 — a release whose assets are the original CRLF versions, so anyone following the README from the top would have re-created the morning's bug exactly. Fixed with the reproducer's two other suggestions in 4b34e28: quick start now pins r4; quickstart/README document where sha256sum -c SHA256SUMS.all must run (the root of a clone of the tag — its entries are repo-relative paths, which is why it fails in the download directory; the reproducer hit this); CONTRIBUTING gains a release checklist whose first rule is "dry-run the published bytes in a clean Linux container" — the exact gap that let CRLF ship. The r4 quickstart.sh asset was re-published with the doc note and recomputed checksums.
2026-10-28site
An external reproducer found the one-liner's real bug in 60 seconds: CRLF line endings — the security model held, the script didn't
The first paid reproduction attempt (Ubuntu 22.04 sandbox, no GPU, script layer only) verified the security model end-to-end: the stub downloaded the wrapper from the pinned release and the SHA-256 check passed — then the wrapper died with set: pipefail: invalid option name. Root cause: every *.sh in the container repo — the run-v1 release asset, the schema-1.3-r4 quickstart.sh asset, this site's stub itself — carried Windows CRLF endings, which bash on Linux reads as pipefail\r. Fixed at every layer: all repo scripts converted to LF with a CI guard that now fails on any CRLF shell script; the wrapper source lives in the repo (run.sh) instead of release-only; a new pinned release run-v2 (release assets are immutable, so run-v1 stays as the tombstone of this bug) and this stub now pins run-v2's checksum; the r4 quickstart.sh asset was re-published as LF with updated checksums (one pre-existing error corrected along the way: SHA256SUMS.all's schema line never matched the tag). A candour note on the word “immutable”: for release assets it is a policy this project keeps by discipline, not a platform guarantee — GitHub assets can in principle be replaced, which is exactly why each re-upload in this entry (the r4 quickstart, twice) is logged here with its recomputed checksums, and why run-v1's assets were left byte-for-byte untouched, its release notes now opening with a superseded banner that points at run-v2; the r3 assets, likewise, were never re-uploaded — the README simply stopped pointing at them. The reproducer's dry-run passes after the line-ending fix and the full GPU reproduction is queued — exactly the loop this project exists for.
2026-10-26site
/spec/ now resolves: the protocol directory page was 404 while being cited as the spec's rendered home
Google Search Console flagged one 404 on this site: /spec/. The cause was self-inflicted and worth recording — the protocol document itself (and the Zenodo record describing it) says the spec is "rendered at quantenergy.tech/spec/", but that directory had no index page, so every crawler following the citation hit a wall. Fixed by adding a real protocol entry page: current version (v1.1) with its canonical DOIs, a map of which layer lives where (protocol / schema / method / changelog), and a note that the Zenodo archive — not any URL — is the canonical citation. The sitemap now lists it. Nothing in the protocol itself changed; this is purely making an already-cited address answer.
2026-10-25site
One-liner run.sh with a hardened security model
quantenergy.tech/run.sh is now a 20-line stub that downloads the real wrapper from the immutable release run-v1 (ecocompute-mlcube), verifies its embedded SHA-256, then executes — a DNS hijack of the site cannot change what runs. The container page documents both paths (four-line audited vs one-liner convenience) under the same pinned-release security model. Homepage, contribute and replications pages link the one-liner.
2026-10-24api
CI gains a daily heartbeat: the deploy-vs-repo checks keep running while the site itself is frozen
The live probes in CI (deployed-Worker drift, /mcp smoke, UA matrix, rate-limit and funnel canaries) were all push-triggered — which means they only ran when the repo changed. The repo is now in a deliberate freeze (no feature work; only responses to new PRs and external replies), so the probes would have gone dark exactly when nobody is watching the code. The workflow now also runs on a daily schedule (01:23 UTC, off the hour to dodge GitHub's scheduler congestion). One line of YAML, no Worker changes, no redeploy — but the "deployed state still matches the repo" defense line no longer depends on the repo being active.
2026-10-24api
MCP calls now feed the first-party funnel — registry-driven distribution is finally measurable
Since the registry listing, assistants can call the estimator over /mcp — but those calls were invisible to the /v1/track funnel, systematically understating distribution. The Worker now records mcp_initialize (client name) and mcp_tool_call (tool, ok, and whitelisted scalars: params_b/arch/precision/batch/ctx) server-side into the same FUNNEL KV, same row shape as the site beacon. Free-text arguments (q, query) are deliberately never stored — the no-PII rule is unchanged. While verifying this, the FUNNEL namespace turned out to be completely empty — funnel events have been silently not landing (cause under investigation; the event TTL was lowered from 400 to 364 days as the prime suspect). CI now runs a funnel canary on every push so this failure class cannot hide again, plus a sustained-rate /mcp probe (~4 req/s for ~40 s) — the per-IP limiter is permissive and eventually consistent per Cloudflare's docs (a sub-second burst passes clean, observed), so it is checked in the regime it actually guards.
2026-10-24api
Custom-domain attempt knocked workers.dev 404 for ~5 minutes; restored, and the UA matrix says the endpoint is client-friendly
Incident: attempting to attach mcp.quantenergy.tech as a Worker Custom Domain failed — the quantenergy.tech zone lives on DNSPod, not in the Cloudflare account, and the partially-applied trigger update took the entire workers.dev route 404 (REST and /mcp, the registry-listed URL) for roughly five minutes until a redeploy without the routes block restored it. CI re-verified after recovery: the REST drift check and all four MCP tools answer correctly again. The wrangler config now carries a warning comment: do not re-add [[routes]] until the zone migrates to Cloudflare. What the outage motivated: a CI UA probe against the deployed /mcp (informational, runs every push) — python-urllib, node, undici, axios, and a generic mcp-client UA all get 200 with a working initialize handshake; only an empty User-Agent is 403'd. So the earlier fear that Cloudflare bot rules on workers.dev might block real MCP clients does not hold in practice — the registry URL is reachable from every realistic client UA we probed.
2026-10-24api
server.json in the repo: the MCP registry listing is now reproducible
api/server.json describes the listing for the official MCP Registry: io.github.hongping-zh/ecocompute-estimator, one streamable-http remote pointing at the live /mcp endpoint. One semantic worth pinning down before it matters: the registry version (1.0.0) is the tool-surface version of the endpoint — deliberately decoupled from the container / dataset versioning. It bumps only when the tool surface changes (a tool added, an output schema changed); the registry uses it to detect updates, and each published version is immutable. The dataset and container keep their own version lines, untouched by registry publishes. Verified against the current registry schema (2025-12-11): remotes[].type = "streamable-http" is the accepted enum value.
2026-10-24site
MCP endpoint: the estimator is now a tool your AI assistant can call
The Worker now speaks MCP (Model Context Protocol) at POST /mcp — stateless streamable HTTP, JSON-RPC 2.0, no key. Four tools: estimate_quantization_energy (ΔE% vs FP16 with basis and confidence), get_quantization_crossover (per-architecture crossover, with the "card + software stack, not architecture constant" warning), list_data_coverage (measured / borrowed / gaps — "no data for your GPU" as a first-class answer), and search_measurements (the raw anchors behind the curves). The design point: every answer embeds its evidence boundaries — basis label, confidence, GPU-package-power scope, run counts, both dataset DOIs — in the same text as the number, so a model relaying it cannot strip the caveats. Announcement post: Make your AI assistant ask about energy first, with Claude Desktop / Cursor config snippets. 16 new Worker tests; REST API and /openapi.json unchanged (the MCP path is listed in both).
2026-10-23site
Estimator-consistency CI: every path the site answers from is now checked on every push
The July contradiction documented in PR #5 (bot-opened 07-26, closed today with thanks) — the homepage headlining a measured −29% while the estimator still answered +20% — was fixed on 07-30 by the July refit, but nothing back then would have failed if it hadn't been. The repo's first CI workflow closes that hole: build/sync_estimate_js.py --check fails when the CURVES literal in estimate.js drifts from curves.json; build/test_headlines.py asserts the homepage's headline claims (7B Ada NF4 saves energy, 7B INT8 costs energy, 3B depends) on both the Python and the JS estimator; optimizer-parity and Worker unit tests run on every push; and a non-blocking live probe compares the deployed Worker's answer against the repo's — first run: both −28.5%, in agreement. The probe needed a browser User-Agent: workers.dev sits behind Cloudflare bot rules that 403 the default Python client.
2026-10-19containersite
Thermal block populated in energy.json; comparability note for context_length
Container: PR #25 populates the thermal block in energy.json — schema fields for thermal state existed since v1.2 but the entrypoint never emitted them. The measurement path now reads GPU temperature via NVML at start/end of measurement and populates temperature_start_c, temperature_end_c, temperature_peak_c (or basis: "unavailable" for AMD/dry-run). New --thermal_mode CLI flag accepts cold (default) or steady. Site: The /contribute/ page now carries a comparability note distinguishing qualified submissions (context_length = 2048, directly comparable to v1.1.0 baseline) from exploratory submissions (context_length ≠ 2048, separate comparability class). Good-first-issue list updated: #18 closed, #24 (thermal block) added.
2026-10-17container
Release schema-1.3-r4: context_length now effective in measurement path
PR #22 makes context_length real: the prompt is tiled/truncated to exactly the recorded value instead of always being ~20 tokens regardless of the parameter. This means runs at context_length ≠ 2048 are not directly comparable with dataset v1.1.0 (whose context_length was nominal). The spec's §8 pin table and all website version matrices have been updated to r4. AMD GPU detection (PR #20) and the backend contributor guide (PR #21) are also included in this release.
2026-10-16container
AMD GPU (ROCm) detection merged; release schema-1.3-r4 includes AMD support
PR #20 (@amarjithkumar2007-wq) adds probe_amd_gpu_name() to entrypoint.py: on a ROCm host the run skips the doomed NVML attempt and reports measurement_source: "ecocompute-dataset (AMD GPU detected, NVML is NVIDIA-only)" with the card name, instead of an opaque NVML error. PR #21 (@yesheng-oss) adds docs/backends.md, a contributor guide for adding llama.cpp / vLLM / TensorRT-LLM as inference backends. Both were merged to main and are included in release schema-1.3-r4 (2026-10-16), which also makes context_length effective in the measurement path (PR #22).
2026-10-16site
Contribution page, contributor recognition, and 404 routing fix
New /contribute/ page with five concrete contribution paths (run a measurement, contribute code, report bugs, improve the protocol, spread the word), linked to four good first issue tickets on the container repo. The About page now carries an Acknowledgments section crediting @gkgoing as the first independent contributor; the Contribute page shows a contributors card near the top. The CONTRIBUTING.md on the container repo was also added. Finally, the 404 alias map was updated: /contribute previously redirected to /replications/#submit — it now points to the real page. Sitemap updated.
2026-10-12site
Sticky section nav, a Chinese landing page with hreflang, a first-party funnel, CSP everywhere, and the quality axis inside the estimator
Six refinements from the final review round: (1) the homepage is long, so a sticky Estimate / Replicate / Audit jump bar now rides
along the top of every scroll position — Estimate switches the tool tab in place, the other two anchor to the verify and cite-audit paths.
(2) The site spoke two languages under one <html lang="en">; now /zh/ is a proper Chinese landing page
(lang="zh-Hans": positioning, three paths, the verified three-line command, conclusion boundaries, the version matrix, six-question FAQ,
links to both Chinese guides), and the EN ↔ ZH pair carries hreflang en / zh-Hans / x-default on both sides, plus a 中文 link in the
top nav and a sitemap entry. (3) The funnel is now recorded end-to-end, first-party, cookie-free: hero CTA clicks, estimator use, container
page arrival, JSON overlay (file, paste and link), deep validation and replication submission all beacon to a new POST /v1/track endpoint
on our own Cloudflare Worker (KV-optional, best-effort, PII-free, covered by tests). (4) GitHub Pages cannot set response headers, so every
page now ships a Content-Security-Policy meta tag, and api/headers-worker.js is the versioned deployment recipe for the full set
(CSP with frame-ancestors, HSTS, X-Content-Type-Options, Permissions-Policy) once the domain is proxied through Cloudflare.
(5) The estimator verdict now states the quality axis next to the energy verdict: NF4 estimates carry the measured +9.5–27.6%
perplexity cost and INT8 the near-zero (+0.5–1.2%) one, both from the RTX 4090 deep dive's same-run pairing — energy was never the whole story.
(6) The first screen now reads "Independent open research project — building toward a reproducible inference-energy standard" instead of
"not a company"; the plain-language explanation stays in the FAQ. No measurement, number or protocol change.
2026-10-11site
Version matrix: one table, three pages — exactly what you are running and citing
The site necessarily speaks in several versions at once — Protocol v1.1, schema ecocompute-energy/1.3,
container release schema-1.3-r4, dataset v1.1.0, the RTX 4090 supplement v3 — and a first-time reader
had to assemble that map alone. Now the same five-row matrix (layer · current version · DOI/release · role, all
links live) sits on the homepage's cite-and-audit path, in the protocol spec's §8 as a "current snapshot" above the
detailed correspondence table, and on the container page as "Where this release sits". The markdown source of the
spec carries the same table. One sentence anchors the whole thing: running the project today means container
release schema-1.3-r4, whose reports validate as ecocompute-energy/1.3 against Protocol v1.1, with every published
number traceable to a dataset DOI. No version, number or measurement changed — this is navigational clarity only.
2026-10-11site
Information architecture: three entry paths, conclusion boundaries beside every verdict, and a self-service replication mechanism
An external review's sharpest point: the homepage was everything at once — paper abstract, dataset readme, protocol,
estimator, optimizer, install guide, Colab, submission flow, changelog and citation, all on one screen. Restructured
without removing any content: (1) the first screen now answers "what can this site do for me" in one line
and three buttons — Estimate my model / Run a real measurement / See the full evidence — followed by a three-path
overview (just want the answer → tool · want to verify it → Colab, container, submission · want to cite
or audit → protocol, DOIs, limitations, changelog). (2) Conclusion-boundary boxes now sit directly beside both
verdicts (full version) and all three core charts (compact): the evidence scope as chips (NVIDIA only · single-stream
decode · batch 1 · 256 tokens · main dataset n = 2 · GPU-package energy · outside-the-fit-is-not-a-measurement)
plus the three-state legend — filled = repeated runs, hollow = a single run, curve = a fit, never a
new measurement. (3) A six-question first-visitor FAQ (paper, tool, dataset or company? no GPU? production? which
role am I?) and the four-layers card moved into a full content index at the bottom.
The replications page becomes a self-service mechanism: a browser-side deep validator (13 checks — schema
version, required blocks, GPU name ↔ declared architecture, sample rate ≥ 10 Hz, thermal state, reference pins,
timestamp, measurement window, image digest, code version, same-session FP16 baseline, outlier range, basis) with an
environment report card (driver, CUDA, torch / transformers / bitsandbytes, package power, thermal), a
paste-JSON alternative to drag-and-drop, and an architecture × precision coverage matrix that shows exactly which
cells of the independent-replication grid are still empty. The honest gaps stay honest: digest and commit hash are
warnings, not failures, because the container does not emit them yet. Both pages now read
replications.json through the assets/data-source.js adapter shipped earlier today. The
four-line download → verify → run command from this morning is untouched, as is every tool, number, filter, the 1/5
goal tracker and the existing submission flow. No measurement, protocol or schema change.
2026-10-11site
Architecture: the seam to a future service layer is now documented — contract only, nothing implemented
An external design review proposed separating the project into a public research layer (this site:
static pages, Apache-2.0/CC-BY code and data, immutable DOIs) and a product/service layer (benchmark
submissions, report generation, private results). We adopted the documentation half and deliberately none of
the implementation: (1) ARCHITECTURE.md defines the boundary — what the research layer will
never gain (logins, billing, private data), the single coupling point between the layers (the
energy.json contract), license and trademark rules, an explicit "not building yet" list, and the
signals (e.g. five independent replications) that would justify starting a service layer at all.
(2) assets/data-source.js is a small adapter so a future page can switch from static JSON to
an API by injecting one line (window.ECO_API_BASE); no page references it yet. (3)
api/openapi.yaml is a contract-only draft of that future API — nothing is deployed, and its
hard rule is that every object must validate against the same public energy.json schema, so private
results stay comparable with the published curves. The estimator Cloudflare Worker that already lives in
api/ is unrelated and untouched. No page behavior, number or measurement changed.
2026-10-11site
curl | bash retired: the quickstart is now a verified release asset, the image runs non-root
An external security review flagged the documented entry point — piping a moving main branch
into a shell — as the site's most urgent fix. Agreed, and fixed at the source
(ecocompute-mlcube schema-1.3-r4),
then on every page that taught the old pattern: the homepage run command, the container page, and both Chinese
guides. The documented way in is now download → verify → run against an immutable release:
curl -fsSLO …/releases/download/schema-1.3-r4/quickstart.sh, curl -fsSLO …/SHA256SUMS,
sha256sum -c SHA256SUMS, bash quickstart.sh. What executes is a specific, checksummed
version — not whatever main holds at the moment you type the command.
The release also carries the rest of the review's list: a CycloneDX SBOM generated from the pinned
requirements.lock.txt; the image digest for pinning instead of the mutable :latest
(sha256:5c812e66…e275, built by CI with the non-root smoke test green); the container now runs
non-root (unprivileged default user; the script has always passed --user $(id -u):$(id -g));
ECOCOMPUTE_READONLY=1 optionally adds --read-only + a tmpfs /tmp, leaving
only the two bind mounts writable; and bash quickstart.sh --dry-run prints the complete plan —
mode, every directory it would touch on your host, the exact commands — and executes nothing. The script header
now states in full what it changes on the host and what it never touches. No measurement, protocol or schema
semantics changed: the validator and the pinned dependency set are byte-identical to schema-1.3-r2.
And one defect found on the way, now corrected: the schema-1.3-r1 pins were computed over Windows
working-tree bytes (CRLF) instead of the repository blobs (LF). Its schema pin never verified against what
raw.githubusercontent.com serves, and its quickstart.sh asset carries CRLF line endings
— which makes bash quickstart.sh fail on Linux at set -euo pipefail. A correction is
appended to the r1
release notes, and the spec's §5.1 pin table now pins r3 —
including the schema's actual LF-blob hash, which the r1 pin had wrong.
2026-10-02site
New note: "Jevons is now a product name" — a commentary on measurement windows in AI pricing claims
Published a commentary note (the blog's exploratory
layer, not counted in the coverage matrix) on
TypeSafe AI's Jev: a vendor claim of 193.6×/444.6× against community-measured medians of 7×/30×, and
the thread through the gap — the chosen measurement window, from "outputs are free" as a pricing policy
to the 70–500 ms latency window that leaves no room for a reasoning tier, at fifty cents per
successful prompt-injection break (Check Point, 25/27). The note draws this site's own discipline into
the argument (NVML package power as a declared window; 30 of 56 coverage-matrix cells empty and
published anyway) and asks four things of API providers: measurement protocols on claims, pricing
windows labeled as policy, published coverage matrices, and easy independent measurement. It contains
no independent measurement of Jev — every number is quoted from the nine public sources listed with
the note. First published in Chinese on Tencent Cloud Developer Community (2026-10-02); this is the
English version. No protocol, data or findings changed.
2026-10-01site
Consistency pass: dataset version in estimator output, GPU-count wording, mobile overflow
A second external review flagged three defects, all verified and fixed. (1) The estimator's JSON
export reported dataset_version: "v1.0" while every page cites v1.1.0; the field is now
generated from a single DATASET_VERSION constant set to v1.1.0. (2) GPU counts
were stated two incompatible ways — "5 GPU architectures" on badges versus "four NVIDIA architectures"
in the BibTeX note. The truth is both at once: 29 anchors span 5 GPU models across 4 architecture classes
(Ada pools the RTX 4090D and RTX 4090). Badges, meta description, the cite page, the evidence page and the
glossary now all use that form. (3) Horizontal overflow at 390px and below: the fitted-curve legend
now wraps, and the mode tabs scroll horizontally instead of clipping at 320px. No numbers changed.
2026-10-01site
Accessibility and SEO pass on the homepage: tabs, landmarks, sitemap
An external review flagged four gaps, all verified and fixed. (1) Tabs are now real ARIA tabs:
the tab buttons carry role="tab" with aria-selected and aria-controls,
each panel is a role="tabpanel", and ArrowLeft/ArrowRight/Home/End switch tabs from the keyboard.
(2) Landmarks: the tool area is wrapped in a <main> with a skip-to-content link as the
first focusable element. (3) The two homepage tools are reachable from the first screen —
"View tested models" and "Estimate your model" CTAs now switch the tab in place instead of only deep-linking
to subpages. (4) Sitemap: /run/, /estimate/ and /optimize/ added
(23 → 26 URLs) and nine stale lastmod values synced to actual commit dates. The meta description
was compressed to ~160 characters and a Referrer-Policy of strict-origin-when-cross-origin added.
Not fixable here, for the record: CSP, HSTS and X-Content-Type-Options are response headers, and
GitHub Pages does not allow custom response headers — they would require a CDN in front of the site. No
numbers or findings changed.
2026-09-30datafinding
Four anchors re-measured on their own stack, two months later: 7B holds to a point, 1.5B does not
The cells the Ada curve leans hardest on — Qwen2-1.5B and Qwen2-7B at NF4 and INT8 — were re-run on
the same software stack they were originally measured on (torch 2.4.1+cu121,
transformers 4.46.3, bitsandbytes 0.45.5, accelerate 1.0.1, the
2026-07-24 configuration), on another rented RTX 4090, two months later. Holding the kernel versions fixed does
not remove the drift — it sorts it by model size. Against the July values (−28.46%, +49.54%, +15.11%,
+180.73%), this session reads: NF4 7B −28.25% (0.2 points away), INT8 7B +50.75%
(1.2 points), NF4 1.5B +22.6% (7.5 points), INT8 1.5B +197.05%
(16.3 points). The two trials on which each cell rests were ~48 minutes apart on one card and
agree to 0.1–4.5 points, so the gap to July is several times the row's own within-session noise.
What this licenses, and what it does not. It licenses saying that the n = 1 risk in this
dataset is concentrated at the small end: at 7B the same configuration returns to about a point across two
months and two rented hosts, while at 1.5B the same configuration can move by more than fifteen points with no
version change to attribute it to — inside a cell whose published range already spans −28% to +199%. It does
not license folding these rows into the published curves, and it does not answer the question left open
on 2026-09-25 (whether the full Ada crossover moves under the current stack) — this session repeated
the July stack, so that question stands.
Published as a separate versioned session, not pooled into the seed counts, the
anchor set or the fitted curves — same policy as every session since 2026-09-20. Data:
data/rtx4090_anchor_2026-09-30.csv
(+ .summary.csv, generator script). Two disclosures travel with it: the reports are
schema-valid but not protocol-conformant — they carry no measurement-window, decoding or arm-order
record (the arm order FP16 → NF4 → INT8 is recovered from timestamps, fixed, not
randomized); and card identity plus the 36 °C start temperature come from one environment snapshot
for the whole session, not per-run logs, so this is within-session trial replication — not across-restart,
and not across cards, replication. No existing number changed. One correction elsewhere on the site comes out
of it: /method/ used to say that no two sessions ran the same torch
build.
Since this one, that is no longer true.
2026-09-28site
The crossover is a card–stack property: attribution wording fixed on every page that states it
An external evaluation caught a real ambiguity: the coverage-matrix notes attributed the NF4 crossing
point to GPU architecture alone (2.1B Turing / 3.7B Ada / 4.8B Blackwell), while the 5090 re-test
section attributed it to software stack alone ("a property of the software stack, not of the
architecture"). Both are half-claims of the same fact, and a reader taking either one page in isolation
could walk away with a wrong model of what moves the crossover. Every page that states the attribution now
says the same thing: the crossover is a property of the card–stack pair — the architecture sets one part
of it, the software stack sets another (Turing/Ada/Blackwell cross at different sizes on a shared-era
stack; on one unchanged RTX 5090, a newer stack moved the crossing from ≈5B to ≈1.8B; the same newer stack
left Ada at break-even). Fixed on: the measured-page grid notes and 5090 section, the methodology page's
finding statement, and the Your-model tab's fit caveats. A second external sweep the same day caught three
residuals on the /cite/ page — the ASCII artifact map still said "a crossover belongs to
a software stack, not to a card", the RTX 5090 BibTeX note still listed "the NF4 crossover moves with the
software stack" as a standalone citable claim (the 2026-09-25 two-card re-test had already falsified that
version — the same newer stack left Ada at break-even), and the copy-ready one-paragraph summary attributed
the crossover to architecture alone. All three now carry the same card–stack-pair framing, and the cite page's
≈1.8B figure is now explicitly tied to the newer stack so it no longer reads as contradicting the matrix's
4.8B. No numbers changed — this is a wording fix that
removes an overclaim in both directions.
2026-09-27protocolsite
Validator hardening: a negative probe that passed as "protocol-conformant" can no longer exist
A hand-built report — non-NVML method, iterations: 1, no window statement,
no decoding record, no arm order, a thermal block with only mode and basis —
sailed through the semantic validator as protocol violations: 0 / protocol-conformant.
That broke the machine-verifiable-core claim, so the validator was rebuilt around one rule:
a report that does not record a protocol-required fact is schema-valid, never
protocol-conformant. Now enforced (release
schema-1.3-r2,
commit 8778c42, validator SHA-256 3dc166d2…10bee3, schema itself unchanged):
NVML method and ≥ 10 Hz (§4.1.1/4.1.2); the FP16-baseline rule covers every non-FP16
precision (§4.2.1); 256 tokens, 10 iterations and greedy decoding must be present and correct
(§4.3.2); the window must be stated as generation (§4.4.1); conformance is floored at schema 1.2+
(§4.5.2); measured thermal blocks need warm-up, cooldown and start/end/peak temperatures, with the
steady temperature excused only by an honest steady_state_reached: false (§4.6.1/4.6.4);
arm order must be recorded or the deviation disclosed (§4.6.2). The probe itself is pinned as a
regression test; the suite is 20/20. Honest consequence: the container does not yet emit the
window/decoding/arm-order records, so its own reports are schema-valid only until it does — same
status as the thermal block, tracked together. §5.1 pins updated.
2026-09-27site
The Run-it-yourself tab writes your issue for you: one-click issue-ready summary
Dropping an energy.json on the Run-it-yourself tab already placed your point
on the crossover curve; the last manual step was writing the GitHub issue by hand. A Copy issue-ready
summary button now does it: the same field table the replications form builds (GPU, model, precision, ΔE%,
basis, iterations, sampling, energy per token, schema_version), a comparison sentence
against the fitted curve (predicted ΔE, 95% band, inside/outside, extrapolation flagged), two attestation
checkboxes to confirm before sending, and the full raw report pasted verbatim in a collapsed block. Clipboard
failure falls back to a visible textarea — the text never depends on browser permissions. The replications page
remains the canonical submission path (handle, n, consent); this is the shortcut from the overlay preview.
2026-09-26siteprotocol
The last spec↔display gaps close: thermal mode on every matrix cell, one replication vocabulary, a stated version of record
Three consistency fixes, each closing a place where the site's display lagged the spec's own rules:
- Coverage matrix cells now carry their thermal mode (
t = c/s/u, next to n).
The spec says compare within a mode, not across one (§4.6.3) — and per §4.6.5 every session behind
the current grid predates the thermal block, so every measured cell is t = u: unknown, not
cold. The grid's footer says so explicitly, the CSV gains a thermal_mode column, and the
September thermal-block re-tests remain deliberately unpooled. When a thermal-block run enters a cell, the
tag changes — a u next to a c is exactly the comparison the spec forbids.
(build/make_coverage_matrix.py,
coverage_matrix_2026-09-09.csv)
- One mapping between the two replication vocabularies. The homepage counts independent
replications (distinct people; 1 / 5), while this page's filled/hollow markers count
measurements per cell (filled = n ≥ 2, hollow = n = 1 — mostly our own repeated
trials). A note beside the matrix now states both definitions and that they never substitute for each
other: a filled cell can still have zero independent replications behind it.
- The /method/ guide now states its version of record: the normative text is the Zenodo record at
DOI 10.5281/zenodo.22958675 (Protocol v1.1) and the guide is synced to that version, linking the
document's §8 protocol ↔ schema ↔ dataset version table. The specification's headings
also gained stable anchor ids, so requirements can be deep-linked (e.g.
§4.6 Thermal state).
2026-09-26protocolsite
Protocol errata: measurement algorithm pinned, arm order a MUST, references pinned by checksum
Four clarifications to the specification, folded into the
site edition (the Zenodo snapshot predates them; they merge into v1.2):
- §4.1 measurement algorithm is now fully specified — trapezoidal integration over the sampler
thread's clock, energy spanning first-to-last sample with no edge extrapolation, dropped reads counted and
reported (fewer than two usable samples ⇒ discard), and the pooled-ratio iteration aggregation
E/token = ΣE/Σtokens. All five items describe exactly what the reference container does.
- §4.6.2 arm order: SHOULD → MUST. Order and thermal state demonstrably move results, so
randomised or counterbalanced order is now mandatory; a fixed order is a disclosed deviation.
- Level C renamed "Standard-candidate" (was "v1.0-grade" — confusing next to protocol v1.1), and
the document status is now "stable core / candidate specification" rather than plain "stable".
- §5.1 pinned artifacts — release tag
schema-1.3-r1, commit, SHA-256 checksums of the
schema and validator, and the image digest, so a conformance claim is a claim against immutable pins, not a
moving main. The container quickstart gains ECOCOMPUTE_REF and a
checksummed release variant replaces curl | bash on main
for reproducible runs.
Site polish in the same pass: the homepage mobile nav is now a proper menu button (nothing truncated),
button time-notes sit below their titles instead of side-by-side, and two absolute phrasings are softened
("Nobody measures…" → "Few open tools measure…"; "Every number is reproducible" → "every published
measurement is accompanied by its reproducibility artifacts").
2026-09-26protocolsite
The spec–schema loop is closed: schema-valid is no longer mistaken for protocol-conformant
The public energy.schema.json checked structure only — a report could pass it while
violating Protocol v1.1 (no thermal block, no same-session FP16 baseline, arbitrary version strings,
sample_rate_hz below 10 Hz). Two layers now close that gap in the
container repo:
the schema itself is tightened (version-enum and const fields, minimum 10 Hz sampling, and at
schema 1.3 the workload/measurement/software MUST-fields become required — no existing valid report
breaks), and a new semantic validator
(ecocompute validate --profile v1.1-core) checks every machine-checkable §4 MUST with
clause-numbered violations and CI exit codes. The compliance levels are renamed to say what they actually
check: Schema-valid / Protocol-conformant / Dataset-eligible (A/B/C unchanged in substance; see
/schema/ §3 and the
spec §5). One honest consequence, stated rather than
hidden: the 2026-09-25 two-card re-test reports themselves grade as schema-valid but not
protocol-conformant — the container does not yet emit the thermal block (tracked issue), and the
validator's test suite pins exactly that verdict.
2026-09-25protocol
The spec layer becomes a standalone, DOI-tracked specification document
Protocol v1.1 is extracted from the method page into an independent normative
document — EcoCompute Protocol v1.1 — written in the
W3C-Note / IETF-RFC style: explicit MUST / SHOULD / MAY requirements for the power source
(NVML GPU-package, ≥ 10 Hz), the same-session FP16 baseline, the workload (batch 1, 256 tokens),
the generation window and the report fields; SHOULD-level replication (n ≥ 2, n travels with the
data); MAY-level extras (additional modes/batches/formats, the draft environment block), each labelled and
never pooled. The document binds the protocol to the machine contract — a conforming submission
MUST pass ecocompute-energy/1.2+ validation — and carries the protocol ↔ schema ↔
dataset version table. The document is archived on Zenodo at
DOI 10.5281/zenodo.22958675 (concept:
10.5281/zenodo.22958674) so papers can cite
"EcoCompute Protocol v1.1, DOI: 10.5281/zenodo.22958675" instead of a moving web page; dataset DOIs and the
protocol DOI stay separate. The
method page remains the human-readable guide and now states that the normative text
is the document. Source: spec/ in the site repository.
2026-09-25siteprotocol
The site is restructured into four layers, and a number bug is fixed
The single mixed homepage is split into Spec (the protocol, normative),
Evidence (everything measured), Changelog (this page) and
Findings (exploratory results, explicitly not counted in the coverage matrix). No existing
URL changed; /changelog/ is the only new one. A number glossary now
defines every count on the site with its denominator, and the homepage hero shows a single number (26 of 56
coverage-matrix cells). Also fixed: the meta description claimed "26 of 56 cells are still empty" — the empty
count is 30; the measured count is 26. Search snippets had been propagating the wrong number.
2026-09-25data
RTX 4090 NF4 re-tested across two physical cards: July's +0.8% anchor holds
Qwen2.5-3B NF4 vs FP16, three independent sessions across two physical RTX 4090s (different UUIDs),
current stack (torch 2.14.0, CUDA 13.0, bitsandbytes 0.50.2 — the same stack as the 2026-09-20 RTX 5090
re-test): +3.5% / +0.7% / +0.5%, mean +1.6%. Conclusions: the July n = 1 +0.8% anchor
is confirmed (Ada 3B is break-even to a small penalty); the −15.1% from the 2026-08-19 paired-quality
session was not reproduced in three current-stack sessions — its stack and conditions differ from the
present run and no specific error was identified, so it is retained as a divergent historical observation and
excluded from current-stack conclusions; and at the 3B anchor the RTX 5090's stack flip
(crossover ≈5B → ≈1.8B) did not appear on the RTX 4090 — only that one anchor was re-measured,
and whether the full Ada crossover curve moves under the current stack awaits the 1.5B and 7B points.
Observed gaps: the two cards differ by ≈ 2.8 points, the two sessions on card 2 by ≈ 0.2 points
(card 1 was measured once, so this is a description, not a variance decomposition; for card-population questions
ncard = 2).
Kept as a separate versioned archive
(data/rtx4090_bnb_2026-09-25.csv,
with power-trace sidecars and the generator script) — not pooled into the seed counts or the fitted curves,
per the same versioning policy as the RTX 5090 re-test. Stack differs from the published pins (bitsandbytes
0.50.2 ≠ 0.43.3), so vs_fp16 is not directly comparable with image runs; the reports say so.
2026-09-24dataprotocol
All energy numbers unified on the generation measurement window
The coverage matrix and every llama.cpp number now state one window: generation only — model load,
on-load quantization and warm-up excluded. The bitsandbytes rows always measured this way (the container starts
its sampler after warm-up; the grid once mislabelled them whole-process), and the llama.cpp GGUF row was
re-cut from its archived 100 Hz power traces: at 576 output tokens it reads
−62.9% on the counter basis (the previously displayed −63.0 was a rounding artefact of the full-precision
−62.9481). Window choice is load-bearing: one session re-integrated over whole-process vs generation vs
decode-only windows moves its 64-token figure by up to 12.5 percentage points (576-token figure: within 2).
Every energy claim now states its window and output length.
Evidence: data/rtx4090_llamacpp_window_comparison_2026-09-24.csv
(generated from the 45 archived traces by build/make_window_comparison.py, assertion-guarded).
2026-09-20datafinding
The RTX 5090 crossover moved from ≈5B to ≈1.8B on a current stack
Re-measuring the same RTX 5090 months later, on bitsandbytes 0.50.2 / torch 2.14 / CUDA 13.0, moved the NF4
break-even size from ≈5B to ≈1.8B — nothing about the hardware changed. A crossover is a property of the
software stack, not only of the architecture. 22 runs, n = 2 across a full instance restart; the two
sessions agree to 2.1 points on average, 5.2 at worst. INT8 stayed positive at every size (+55% at 7B, +256% at
0.5B). A torchao FP8 companion sweep found both FP8 code paths (weight-only, dynamic activation+weight) more
expensive than FP16 at every size, failing in opposite directions.
Maintained as a separate archive — not pooled into the seed counts or the fitted
curves, which still describe the 0.4x-era stack: pooling the two would produce a crossover neither session
measured. Data: rtx5090_bnb_2026-09-20.csv,
rtx5090_fp8_torchao_2026-09-20.csv;
raw reports archived at 10.5281/zenodo.22855133.
2026-09-03finding
The kernel decides the sign, not the bit width
Inside one runtime on one RTX 4090: llama.cpp GGUF Q4_0 decoding Llama-3.1-8B-Instruct costs
61.9% less energy per token than GGUF F16 (for +5.60% perplexity) — the quantized arms draw
more power (296–319 W vs 273 W) and win purely on throughput. On the same card, bitsandbytes
LLM.int8() costs 106% more. Same idea, opposite sign: format-level claims ("4-bit saves
energy") are meaningless without the kernel. Corrected the same day over 45 runs (n = 5, randomized
order, cooldown before every run, hardware energy counter): the provisional −63.6% was an overestimate.
Note: the note and raw data
(10.5281/zenodo.22295184)
predate the 2026-09-24 window unification; where the two disagree on a number, the window-comparison CSV is
authoritative.
Older history — the v1.1.0 dataset release, the July RTX 4090 deep dive, the August INT8 repeats —
is documented in the notes and in each Zenodo record's release notes, not reconstructed here.